Note: This is test shard 1 of 3.
[==========] Running 2 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 2 tests from CatalogManagerConfigurations/MasterStressTest
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260927 23:27:41.065730 2276 test_util.cc:268] Using random seed: 1270199656
2026-09-27T23:27:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:27:41Z Disabled control of system clock
I20260927 23:27:41.121218 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.62:38773
--webserver_interface=127.2.57.62
--webserver_port=0
--builtin_ntp_servers=127.2.57.20:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260927 23:27:41.443437 2285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:27:41.444116 2285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:27:41.444614 2285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:27:41.477916 2285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:27:41.478247 2285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:27:41.478466 2285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:27:41.478662 2285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:27:41.516424 2285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.62:38773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.57.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:27:41.517947 2285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:27:41.519788 2285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:27:41.539314 2292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:27:41.539348 2291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:27:41.540452 2294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:41.541450 2285 server_base.cc:1034] running on GCE node
I20260927 23:27:42.651839 2285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:27:42.655361 2285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:27:42.656844 2285 hybrid_clock.cc:648] HybridClock initialized: now 1790551662656813 us; error 53 us; skew 500 ppm
I20260927 23:27:42.657822 2285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:27:42.665343 2285 webserver.cc:460] Webserver started at http://127.2.57.62:41409/ using document root <none> and password file <none>
I20260927 23:27:42.666482 2285 fs_manager.cc:359] Metadata directory not provided
I20260927 23:27:42.666714 2285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:27:42.667201 2285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:27:42.672945 2285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/data/instance:
uuid: "bee21c28e51643189178a3c851672546"
format_stamp: "Formatted at 2026-09-27 23:27:42 on dist-test-slave-jbx2"
I20260927 23:27:42.674264 2285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/wal/instance:
uuid: "bee21c28e51643189178a3c851672546"
format_stamp: "Formatted at 2026-09-27 23:27:42 on dist-test-slave-jbx2"
I20260927 23:27:42.682932 2285 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260927 23:27:42.689903 2301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:27:42.691289 2285 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.002s
I20260927 23:27:42.691687 2285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/wal
uuid: "bee21c28e51643189178a3c851672546"
format_stamp: "Formatted at 2026-09-27 23:27:42 on dist-test-slave-jbx2"
I20260927 23:27:42.692152 2285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:27:42.740988 2285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:27:42.742753 2285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:27:42.743278 2285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:27:42.822587 2285 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.62:38773
I20260927 23:27:42.822683 2352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.62:38773 every 8 connection(s)
I20260927 23:27:42.825642 2285 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/data/info.pb
I20260927 23:27:42.831616 2353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:42.834447 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 2285
I20260927 23:27:42.835029 2276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-0/wal/instance
I20260927 23:27:42.840971 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.61:34987
--webserver_interface=127.2.57.61
--webserver_port=0
--builtin_ntp_servers=127.2.57.20:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260927 23:27:42.848577 2353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:27:42.883754 2353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } has no permanent_uuid. Determining permanent_uuid...
W20260927 23:27:42.886381 2302 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.61:34987: connect: Connection refused (error 111)
W20260927 23:27:42.890717 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:34987: Network error: Client connection negotiation failed: client connection to 127.2.57.61:34987: connect: Connection refused (error 111)
I20260927 23:27:42.940397 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } attempt: 1
W20260927 23:27:42.944712 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:34987: Network error: Client connection negotiation failed: client connection to 127.2.57.61:34987: connect: Connection refused (error 111)
I20260927 23:27:43.013213 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } attempt: 2
W20260927 23:27:43.017756 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:34987: Network error: Client connection negotiation failed: client connection to 127.2.57.61:34987: connect: Connection refused (error 111)
I20260927 23:27:43.109236 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } attempt: 3
W20260927 23:27:43.113726 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:34987: Network error: Client connection negotiation failed: client connection to 127.2.57.61:34987: connect: Connection refused (error 111)
W20260927 23:27:43.192027 2355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:27:43.192773 2355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:27:43.193310 2355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:27:43.227101 2355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:27:43.227530 2355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:27:43.227821 2355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:27:43.228066 2355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:27:43.257153 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } attempt: 4
W20260927 23:27:43.261652 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:34987: Network error: Client connection negotiation failed: client connection to 127.2.57.61:34987: connect: Connection refused (error 111)
I20260927 23:27:43.269821 2355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.61:34987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.57.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:27:43.271433 2355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:27:43.273561 2355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:27:43.293471 2367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:43.561142 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } attempt: 5
W20260927 23:27:43.565551 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:34987: Network error: Client connection negotiation failed: client connection to 127.2.57.61:34987: connect: Connection refused (error 111)
I20260927 23:27:44.112973 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } attempt: 6
W20260927 23:27:44.122776 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:34987: Network error: Client connection negotiation failed: client connection to 127.2.57.61:34987: connect: Connection refused (error 111)
W20260927 23:27:43.293476 2364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:43.295550 2355 server_base.cc:1034] running on GCE node
W20260927 23:27:43.293684 2365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:44.420676 2355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:27:44.423799 2355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:27:44.425197 2355 hybrid_clock.cc:648] HybridClock initialized: now 1790551664425153 us; error 71 us; skew 500 ppm
I20260927 23:27:44.426122 2355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:27:44.433425 2355 webserver.cc:460] Webserver started at http://127.2.57.61:45757/ using document root <none> and password file <none>
I20260927 23:27:44.434530 2355 fs_manager.cc:359] Metadata directory not provided
I20260927 23:27:44.434769 2355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:27:44.435293 2355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:27:44.441874 2355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/data/instance:
uuid: "19a88ff0709d4266b933cc526577127f"
format_stamp: "Formatted at 2026-09-27 23:27:44 on dist-test-slave-jbx2"
I20260927 23:27:44.443472 2355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/wal/instance:
uuid: "19a88ff0709d4266b933cc526577127f"
format_stamp: "Formatted at 2026-09-27 23:27:44 on dist-test-slave-jbx2"
I20260927 23:27:44.452171 2355 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.010s sys 0.001s
I20260927 23:27:44.458801 2376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:27:44.460078 2355 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20260927 23:27:44.460498 2355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/wal
uuid: "19a88ff0709d4266b933cc526577127f"
format_stamp: "Formatted at 2026-09-27 23:27:44 on dist-test-slave-jbx2"
I20260927 23:27:44.460875 2355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:27:44.504940 2355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:27:44.506660 2355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:27:44.507184 2355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:27:44.594318 2355 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.61:34987
I20260927 23:27:44.594419 2427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.61:34987 every 8 connection(s)
I20260927 23:27:44.598553 2355 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/data/info.pb
I20260927 23:27:44.604789 2428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:44.606657 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 2355
I20260927 23:27:44.607168 2276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-1/wal/instance
I20260927 23:27:44.610666 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.60:41571
--webserver_interface=127.2.57.60
--webserver_port=0
--builtin_ntp_servers=127.2.57.20:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260927 23:27:44.621089 2428 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:27:44.646907 2428 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:27:44.663859 2428 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } has no permanent_uuid. Determining permanent_uuid...
W20260927 23:27:44.666608 2378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
W20260927 23:27:44.671092 2428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
I20260927 23:27:44.720685 2428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 1
W20260927 23:27:44.724874 2428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
I20260927 23:27:44.793437 2428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 2
W20260927 23:27:44.797657 2428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
I20260927 23:27:44.889158 2428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 3
W20260927 23:27:44.893481 2428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
W20260927 23:27:44.948613 2430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:27:44.949314 2430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:27:44.949754 2430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:27:44.983762 2430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:27:44.984119 2430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:27:44.984371 2430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:27:44.984573 2430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:27:45.023917 2430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.60:41571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.57.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:27:45.025539 2430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:27:45.027441 2430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 23:27:45.036926 2428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 4
W20260927 23:27:45.042073 2428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
W20260927 23:27:45.049700 2443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:45.183398 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } attempt: 7
I20260927 23:27:45.204823 2353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } has no permanent_uuid. Determining permanent_uuid...
W20260927 23:27:45.211036 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
I20260927 23:27:45.269564 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 1
W20260927 23:27:45.274293 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
I20260927 23:27:45.341713 2428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 5
W20260927 23:27:45.346719 2428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
I20260927 23:27:45.355803 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 2
W20260927 23:27:45.360364 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
I20260927 23:27:45.445972 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 3
W20260927 23:27:45.450421 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
I20260927 23:27:45.591001 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 4
W20260927 23:27:45.595117 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
I20260927 23:27:45.878620 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 5
W20260927 23:27:45.882845 2353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
I20260927 23:27:45.894253 2428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 6
W20260927 23:27:45.902215 2428 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:41571: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
W20260927 23:27:45.049779 2441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:45.051496 2430 server_base.cc:1034] running on GCE node
W20260927 23:27:45.049865 2440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:46.195544 2430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:27:46.198904 2430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:27:46.200425 2430 hybrid_clock.cc:648] HybridClock initialized: now 1790551666200364 us; error 61 us; skew 500 ppm
I20260927 23:27:46.201413 2430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:27:46.208884 2430 webserver.cc:460] Webserver started at http://127.2.57.60:39731/ using document root <none> and password file <none>
I20260927 23:27:46.209976 2430 fs_manager.cc:359] Metadata directory not provided
I20260927 23:27:46.210238 2430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:27:46.210768 2430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:27:46.216367 2430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data/instance:
uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150"
format_stamp: "Formatted at 2026-09-27 23:27:46 on dist-test-slave-jbx2"
I20260927 23:27:46.217757 2430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/wal/instance:
uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150"
format_stamp: "Formatted at 2026-09-27 23:27:46 on dist-test-slave-jbx2"
I20260927 23:27:46.226030 2430 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s
I20260927 23:27:46.232731 2455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:27:46.234040 2430 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:27:46.234476 2430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/wal
uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150"
format_stamp: "Formatted at 2026-09-27 23:27:46 on dist-test-slave-jbx2"
I20260927 23:27:46.234877 2430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:27:46.279517 2430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:27:46.281401 2430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:27:46.281937 2430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:27:46.361531 2430 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.60:41571
I20260927 23:27:46.361601 2506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.60:41571 every 8 connection(s)
I20260927 23:27:46.364683 2430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data/info.pb
I20260927 23:27:46.365581 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 2430
I20260927 23:27:46.366025 2276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/wal/instance
I20260927 23:27:46.371899 2507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:46.392166 2507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:27:46.419706 2507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:27:46.433187 2507 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:27:46.435451 2353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 6
I20260927 23:27:46.459457 2507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: Bootstrap starting.
I20260927 23:27:46.461011 2353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546: Bootstrap starting.
I20260927 23:27:46.467034 2353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:46.467693 2507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:46.470010 2353 log.cc:824] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546: Log is configured to *not* fsync() on all Append() calls
I20260927 23:27:46.470287 2507 log.cc:824] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: Log is configured to *not* fsync() on all Append() calls
I20260927 23:27:46.476131 2507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: No bootstrap required, opened a new log
I20260927 23:27:46.476094 2353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546: No bootstrap required, opened a new log
I20260927 23:27:46.509022 2507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:46.509017 2353 raft_consensus.cc:348] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:46.509971 2507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:46.509974 2353 raft_consensus.cc:374] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:46.510290 2353 raft_consensus.cc:729] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bee21c28e51643189178a3c851672546, State: Initialized, Role: FOLLOWER
I20260927 23:27:46.510285 2507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cdd64d48a684d5ebccc2e2e8a4e3150, State: Initialized, Role: FOLLOWER
I20260927 23:27:46.511281 2507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:46.511283 2353 consensus_queue.cc:260] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:46.514905 2520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:46.515842 2520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:27:46.517508 2507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:27:46.517475 2521 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:46.518625 2521 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:27:46.519632 2353 sys_catalog.cc:564] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: configured and running, proceeding with master startup.
W20260927 23:27:46.537873 2461 tablet.cc:2369] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 23:27:46.563961 2542 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:27:46.564527 2542 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:27:46.581529 2543 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:27:46.582011 2543 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:27:46.554631 2285 master_runner.cc:186] Error getting master registration for 127.2.57.61:34987: OK, instance_id { permanent_uuid: "19a88ff0709d4266b933cc526577127f" instance_seqno: 1790551664566899 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 23:27:46.600008 2285 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 23:27:46.567937 2430 master_runner.cc:186] Error getting master registration for 127.2.57.61:34987: OK, instance_id { permanent_uuid: "19a88ff0709d4266b933cc526577127f" instance_seqno: 1790551664566899 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 23:27:46.607291 2430 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 23:27:46.654336 2521 raft_consensus.cc:482] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:46.654955 2521 raft_consensus.cc:504] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:46.659114 2521 leader_election.cc:288] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:27:46.659116 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" is_pre_election: true
I20260927 23:27:46.660254 2482 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bee21c28e51643189178a3c851672546 in term 0.
I20260927 23:27:46.661788 2303 leader_election.cc:302] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150, bee21c28e51643189178a3c851672546; no voters:
I20260927 23:27:46.662840 2521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:46.663199 2521 raft_consensus.cc:482] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:46.663806 2521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:46.671139 2521 raft_consensus.cc:504] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:46.682937 2521 leader_election.cc:288] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 1 election: Requested vote from peers 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:27:46.683137 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150"
I20260927 23:27:46.683756 2482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:46.691197 2482 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bee21c28e51643189178a3c851672546 in term 1.
I20260927 23:27:46.692435 2303 leader_election.cc:302] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150, bee21c28e51643189178a3c851672546; no voters:
I20260927 23:27:46.693163 2521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:46.696178 2521 raft_consensus.cc:686] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 1 LEADER]: Becoming Leader. State: Replica: bee21c28e51643189178a3c851672546, State: Running, Role: LEADER
I20260927 23:27:46.697438 2521 consensus_queue.cc:237] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:46.710693 2546 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bee21c28e51643189178a3c851672546. Latest consensus state: current_term: 1 leader_uuid: "bee21c28e51643189178a3c851672546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:46.711606 2546 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: This master's current role is: LEADER
I20260927 23:27:46.715057 2548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:27:46.723212 2548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:27:46.745893 2482 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 1 FOLLOWER]: Refusing update from remote peer bee21c28e51643189178a3c851672546: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:27:46.747686 2546 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:27:46.754680 2307 tablet.cc:2369] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:27:46.781520 2520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bee21c28e51643189178a3c851672546. Latest consensus state: current_term: 1 leader_uuid: "bee21c28e51643189178a3c851672546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:46.783802 2520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:27:46.801221 2521 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bee21c28e51643189178a3c851672546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:46.802130 2521 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: This master's current role is: LEADER
I20260927 23:27:46.803303 2520 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bee21c28e51643189178a3c851672546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:46.804100 2520 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:27:46.810995 2548 catalog_manager.cc:1347] Generated new cluster ID: 994f6841308149a98b641f5878f9d494
I20260927 23:27:46.811360 2548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:27:46.859705 2548 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:27:46.862768 2548 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:27:46.895780 2548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546: Generated new TSK 0
I20260927 23:27:46.897291 2548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:27:46.927700 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.57.1:0
--local_ip_for_outbound_sockets=127.2.57.1
--webserver_interface=127.2.57.1
--webserver_port=0
--tserver_master_addrs=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--builtin_ntp_servers=127.2.57.20:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260927 23:27:46.962560 2428 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } attempt: 7
I20260927 23:27:47.007995 2428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Bootstrap starting.
I20260927 23:27:47.027624 2428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:47.034847 2428 log.cc:824] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Log is configured to *not* fsync() on all Append() calls
I20260927 23:27:47.059362 2428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: No bootstrap required, opened a new log
I20260927 23:27:47.143488 2428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:47.146181 2428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:47.147133 2428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19a88ff0709d4266b933cc526577127f, State: Initialized, Role: FOLLOWER
I20260927 23:27:47.149806 2428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:47.175438 2403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:27:47.176467 2403 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bee21c28e51643189178a3c851672546 in term 0.
W20260927 23:27:47.177075 2403 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45290 (request call id 2) took 518 ms (client timeout 506 ms). Trace:
W20260927 23:27:47.177420 2403 rpcz_store.cc:269] 0927 23:27:46.658282 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:46.658466 (+ 184us) service_pool.cc:227] Handling call
0927 23:27:47.176989 (+518523us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:27:47.178916 2402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f"
I20260927 23:27:47.179603 2402 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:47.188220 2402 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bee21c28e51643189178a3c851672546 in term 1.
I20260927 23:27:47.189826 2401 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Refusing update from remote peer bee21c28e51643189178a3c851672546: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:27:47.191385 2403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:27:47.192340 2403 raft_consensus.cc:2362] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate bee21c28e51643189178a3c851672546 in term 1. Re-sending same reply.
W20260927 23:27:47.197819 2402 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45290 (request call id 3) took 516 ms (client timeout 617 ms). Trace:
I20260927 23:27:47.197727 2546 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:27:47.203359 2402 rpcz_store.cc:269] 0927 23:27:46.681483 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:46.681626 (+ 143us) service_pool.cc:227] Handling call
0927 23:27:47.197752 (+516126us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:27:47.199142 2428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:27:47.174264 2558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:47.209259 2558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:27:47.266731 2558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bee21c28e51643189178a3c851672546. Latest consensus state: current_term: 1 leader_uuid: "bee21c28e51643189178a3c851672546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:47.267838 2558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:27:47.267621 2560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bee21c28e51643189178a3c851672546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:47.268497 2560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:27:47.333679 2575 catalog_manager.cc:1259] Loaded cluster ID: 994f6841308149a98b641f5878f9d494
I20260927 23:27:47.334021 2575 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: loading cluster ID for follower catalog manager: success
I20260927 23:27:47.352607 2575 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: acquiring CA information for follower catalog manager: success
I20260927 23:27:47.373116 2575 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 23:27:47.506891 2554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:27:47.509589 2554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:27:47.510622 2554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260927 23:27:47.574566 2542 catalog_manager.cc:1259] Loaded cluster ID: 994f6841308149a98b641f5878f9d494
I20260927 23:27:47.575019 2542 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: loading cluster ID for follower catalog manager: success
W20260927 23:27:47.587723 2554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
I20260927 23:27:47.592859 2542 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: acquiring CA information for follower catalog manager: success
I20260927 23:27:47.599409 2542 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 23:27:47.601351 2554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.57.1
I20260927 23:27:47.668380 2554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.57.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.57.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.57.1
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:27:47.670580 2554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:27:47.673251 2554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 23:27:47.711205 2560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bee21c28e51643189178a3c851672546)
I20260927 23:27:47.711764 2560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:47.715847 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19a88ff0709d4266b933cc526577127f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:27:47.719746 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19a88ff0709d4266b933cc526577127f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" is_pre_election: true
I20260927 23:27:47.721827 2378 leader_election.cc:302] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19a88ff0709d4266b933cc526577127f; no voters: 0cdd64d48a684d5ebccc2e2e8a4e3150, bee21c28e51643189178a3c851672546
I20260927 23:27:47.722856 2558 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:27:47.722965 2560 leader_election.cc:288] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
W20260927 23:27:47.747061 2582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:47.827382 2546 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bee21c28e51643189178a3c851672546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:47.828445 2546 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: This master's current role is: LEADER
I20260927 23:27:48.348218 2591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bee21c28e51643189178a3c851672546)
I20260927 23:27:48.349346 2591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:48.353237 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19a88ff0709d4266b933cc526577127f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:27:48.361093 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19a88ff0709d4266b933cc526577127f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" is_pre_election: true
I20260927 23:27:48.363531 2378 leader_election.cc:302] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19a88ff0709d4266b933cc526577127f; no voters: 0cdd64d48a684d5ebccc2e2e8a4e3150, bee21c28e51643189178a3c851672546
I20260927 23:27:48.372754 2591 leader_election.cc:288] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:27:48.374531 2592 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:27:49.293632 2594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bee21c28e51643189178a3c851672546)
I20260927 23:27:49.295374 2593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bee21c28e51643189178a3c851672546)
I20260927 23:27:49.296111 2594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:49.304581 2593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:49.307624 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19a88ff0709d4266b933cc526577127f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:27:49.314177 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19a88ff0709d4266b933cc526577127f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" is_pre_election: true
I20260927 23:27:49.314944 2482 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19a88ff0709d4266b933cc526577127f in term 1.
I20260927 23:27:49.316413 2378 leader_election.cc:302] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150, 19a88ff0709d4266b933cc526577127f; no voters: bee21c28e51643189178a3c851672546
I20260927 23:27:49.321478 2594 leader_election.cc:288] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:27:49.325091 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:27:49.327669 2594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 23:27:49.329989 2594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Starting leader election (detected failure of leader bee21c28e51643189178a3c851672546)
I20260927 23:27:49.331004 2594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 1 FOLLOWER]: Advancing to term 2
I20260927 23:27:49.333487 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:27:49.336632 2456 leader_election.cc:302] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150; no voters: 19a88ff0709d4266b933cc526577127f, bee21c28e51643189178a3c851672546
I20260927 23:27:49.339807 2593 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987)
I20260927 23:27:49.348727 2593 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:27:49.369320 2378 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.60:41571, user_credentials={real_user=slave}} blocked reactor thread for 53068.6us
I20260927 23:27:49.374007 2594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:49.379259 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19a88ff0709d4266b933cc526577127f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546"
I20260927 23:27:49.386127 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19a88ff0709d4266b933cc526577127f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150"
I20260927 23:27:49.388067 2378 leader_election.cc:302] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 19a88ff0709d4266b933cc526577127f; no voters: 0cdd64d48a684d5ebccc2e2e8a4e3150, bee21c28e51643189178a3c851672546
I20260927 23:27:49.389065 2595 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260927 23:27:49.389873 2594 leader_election.cc:288] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [CANDIDATE]: Term 2 election: Requested vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:27:49.889801 2401 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 2 FOLLOWER]: Rejecting Update request from peer bee21c28e51643189178a3c851672546 for earlier term 1. Current term is 2. Ops: []
I20260927 23:27:49.891657 2588 consensus_queue.cc:1046] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260927 23:27:49.896412 2597 raft_consensus.cc:3032] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 1 LEADER]: Stepping down as leader of term 1
I20260927 23:27:49.898026 2597 raft_consensus.cc:729] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bee21c28e51643189178a3c851672546, State: Running, Role: LEADER
I20260927 23:27:49.899693 2597 consensus_queue.cc:260] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:49.915390 2597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 1 FOLLOWER]: Advancing to term 2
I20260927 23:27:49.920680 2598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:49.923484 2598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:49.932832 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19a88ff0709d4266b933cc526577127f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:27:49.936200 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19a88ff0709d4266b933cc526577127f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" is_pre_election: true
I20260927 23:27:49.941875 2598 leader_election.cc:288] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:27:49.950140 2328 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19a88ff0709d4266b933cc526577127f in term 2.
I20260927 23:27:49.956761 2379 leader_election.cc:302] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19a88ff0709d4266b933cc526577127f, bee21c28e51643189178a3c851672546; no voters: 0cdd64d48a684d5ebccc2e2e8a4e3150
I20260927 23:27:49.958179 2598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260927 23:27:49.958621 2598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:49.958994 2598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 2 FOLLOWER]: Advancing to term 3
W20260927 23:27:49.104346 2581 debug-util.cc:398] Leaking SignalData structure 0x7b08000148c0 after lost signal to thread 2554
W20260927 23:27:49.961059 2581 kernel_stack_watchdog.cc:198] Thread 2554 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 365ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260927 23:27:47.749135 2583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:27:49.965752 2554 thread.cc:636] OpenStack (cloud detector) Time spent creating pthread: real 2.229s user 0.662s sys 0.829s
W20260927 23:27:49.966257 2554 thread.cc:602] OpenStack (cloud detector) Time spent starting thread: real 2.230s user 0.662s sys 0.829s
I20260927 23:27:49.966569 2554 server_base.cc:1034] running on GCE node
W20260927 23:27:49.967710 2585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:49.969483 2554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:27:49.972458 2598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:49.979292 2554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:27:49.983009 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19a88ff0709d4266b933cc526577127f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546"
I20260927 23:27:49.983690 2328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 2 FOLLOWER]: Advancing to term 3
I20260927 23:27:49.984974 2554 hybrid_clock.cc:648] HybridClock initialized: now 1790551669984842 us; error 134 us; skew 500 ppm
I20260927 23:27:49.986223 2554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:27:49.999207 2328 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19a88ff0709d4266b933cc526577127f in term 3.
I20260927 23:27:50.001762 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "19a88ff0709d4266b933cc526577127f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150"
I20260927 23:27:50.005733 2554 webserver.cc:460] Webserver started at http://127.2.57.1:33429/ using document root <none> and password file <none>
I20260927 23:27:50.007359 2554 fs_manager.cc:359] Metadata directory not provided
I20260927 23:27:50.007694 2554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:27:50.008411 2554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:27:50.009634 2598 leader_election.cc:288] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [CANDIDATE]: Term 3 election: Requested vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:27:50.011205 2379 leader_election.cc:302] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 19a88ff0709d4266b933cc526577127f, bee21c28e51643189178a3c851672546; no voters: 0cdd64d48a684d5ebccc2e2e8a4e3150
I20260927 23:27:50.016793 2554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/data/instance:
uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
format_stamp: "Formatted at 2026-09-27 23:27:50 on dist-test-slave-jbx2"
I20260927 23:27:50.017182 2598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 3 FOLLOWER]: Leader election won for term 3
I20260927 23:27:50.018663 2554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/instance:
uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
format_stamp: "Formatted at 2026-09-27 23:27:50 on dist-test-slave-jbx2"
I20260927 23:27:50.028939 2598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 3 LEADER]: Becoming Leader. State: Replica: 19a88ff0709d4266b933cc526577127f, State: Running, Role: LEADER
I20260927 23:27:50.030362 2598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:50.035503 2554 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.008s sys 0.008s
I20260927 23:27:50.053845 2602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19a88ff0709d4266b933cc526577127f. Latest consensus state: current_term: 3 leader_uuid: "19a88ff0709d4266b933cc526577127f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:50.054673 2602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: This master's current role is: LEADER
I20260927 23:27:50.056166 2605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:27:50.057849 2554 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.001s sys 0.005s
I20260927 23:27:50.058296 2554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal
uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
format_stamp: "Formatted at 2026-09-27 23:27:50 on dist-test-slave-jbx2"
I20260927 23:27:50.058792 2554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:27:50.061623 2606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:27:50.074724 2606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:27:50.079664 2606 catalog_manager.cc:1259] Loaded cluster ID: 994f6841308149a98b641f5878f9d494
I20260927 23:27:50.080231 2606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:27:50.085759 2606 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:27:50.090251 2606 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Loaded TSK: 0
I20260927 23:27:50.092132 2606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:27:50.114753 2554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:27:50.117005 2554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:27:50.117637 2554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:27:50.121433 2554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:27:50.127781 2554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:27:50.128108 2554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:27:50.128463 2554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:27:50.128677 2554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:27:50.286623 2685 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bee21c28e51643189178a3c851672546)
I20260927 23:27:50.287551 2685 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:50.290833 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:27:50.291676 2328 raft_consensus.cc:2354] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0cdd64d48a684d5ebccc2e2e8a4e3150 for earlier term 2. Current term is 3.
I20260927 23:27:50.293557 2458 leader_election.cc:398] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 2 pre-election: Vote denied by peer bee21c28e51643189178a3c851672546 (127.2.57.62:38773) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0cdd64d48a684d5ebccc2e2e8a4e3150 for earlier term 2. Current term is 3.
I20260927 23:27:50.294031 2458 leader_election.cc:401] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260927 23:27:50.297480 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:27:50.299974 2685 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987)
I20260927 23:27:50.301013 2685 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 1 FOLLOWER]: Advancing to term 3
I20260927 23:27:50.314687 2685 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer bee21c28e51643189178a3c851672546 (127.2.57.62:38773) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0cdd64d48a684d5ebccc2e2e8a4e3150 for earlier term 2. Current term is 3.
I20260927 23:27:50.412377 2554 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.1:35657
I20260927 23:27:50.412420 2720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.1:35657 every 8 connection(s)
I20260927 23:27:50.416579 2554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/data/info.pb
I20260927 23:27:50.419868 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 2554
I20260927 23:27:50.420532 2276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/instance
I20260927 23:27:50.446110 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.57.2:0
--local_ip_for_outbound_sockets=127.2.57.2
--webserver_interface=127.2.57.2
--webserver_port=0
--tserver_master_addrs=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--builtin_ntp_servers=127.2.57.20:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260927 23:27:50.495818 2722 heartbeater.cc:347] Connected to a master server at 127.2.57.61:34987
I20260927 23:27:50.496428 2722 heartbeater.cc:464] Registering TS with master...
I20260927 23:27:50.497869 2722 heartbeater.cc:511] Master 127.2.57.61:34987 requested a full tablet report, sending...
I20260927 23:27:50.501735 2393 ts_manager.cc:194] Registered new tserver with Master: 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:50.502489 2721 heartbeater.cc:347] Connected to a master server at 127.2.57.60:41571
I20260927 23:27:50.502838 2721 heartbeater.cc:464] Registering TS with master...
I20260927 23:27:50.502765 2328 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Refusing update from remote peer 19a88ff0709d4266b933cc526577127f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260927 23:27:50.503680 2721 heartbeater.cc:511] Master 127.2.57.60:41571 requested a full tablet report, sending...
I20260927 23:27:50.504768 2602 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260927 23:27:50.505127 2393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.57.1:39229
I20260927 23:27:50.507004 2472 ts_manager.cc:194] Registered new tserver with Master: 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:50.531095 2725 heartbeater.cc:347] Connected to a master server at 127.2.57.62:38773
I20260927 23:27:50.531526 2725 heartbeater.cc:464] Registering TS with master...
I20260927 23:27:50.532333 2725 heartbeater.cc:511] Master 127.2.57.62:38773 requested a full tablet report, sending...
I20260927 23:27:50.536047 2318 ts_manager.cc:194] Registered new tserver with Master: 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:50.545532 2732 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19a88ff0709d4266b933cc526577127f. Latest consensus state: current_term: 3 leader_uuid: "19a88ff0709d4266b933cc526577127f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:50.548863 2732 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:27:50.560736 2598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "19a88ff0709d4266b933cc526577127f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:50.561506 2598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: This master's current role is: LEADER
I20260927 23:27:50.562057 2482 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Refusing update from remote peer 19a88ff0709d4266b933cc526577127f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260927 23:27:50.564582 2732 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "19a88ff0709d4266b933cc526577127f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:50.564867 2602 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260927 23:27:50.565588 2732 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:27:50.574195 2685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19a88ff0709d4266b933cc526577127f. Latest consensus state: current_term: 3 leader_uuid: "19a88ff0709d4266b933cc526577127f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:50.575011 2685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:27:50.574998 2688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "19a88ff0709d4266b933cc526577127f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:50.575762 2688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:27:50.590327 2543 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 23:27:50.861177 2731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:27:50.861800 2731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:27:50.862483 2731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:27:50.896212 2731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:27:50.897276 2731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.57.2
I20260927 23:27:50.935506 2731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.57.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.57.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.57.2
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:27:50.937095 2731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:27:50.939342 2731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 23:27:51.012229 2722 heartbeater.cc:503] Master 127.2.57.61:34987 was elected leader, sending a full tablet report...
I20260927 23:27:51.068454 2688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:27:51.069181 2688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:51.072117 2688 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987)
I20260927 23:27:51.072531 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:27:51.072808 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:27:51.074709 2458 leader_election.cc:302] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150; no voters: 19a88ff0709d4266b933cc526577127f, bee21c28e51643189178a3c851672546
I20260927 23:27:51.075421 2688 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:27:51.107909 2602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "19a88ff0709d4266b933cc526577127f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:51.108845 2602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: This master's current role is: LEADER
I20260927 23:27:51.617019 2754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:27:51.617594 2754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:51.620453 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:27:51.622982 2754 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987)
I20260927 23:27:51.624477 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:27:51.626755 2456 leader_election.cc:302] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150; no voters: 19a88ff0709d4266b933cc526577127f, bee21c28e51643189178a3c851672546
I20260927 23:27:51.627662 2754 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:27:50.963546 2731 server_base.cc:1034] running on GCE node
W20260927 23:27:50.961795 2744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:27:50.961833 2746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:27:50.961771 2743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:52.097380 2731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:27:52.100272 2731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:27:52.101773 2731 hybrid_clock.cc:648] HybridClock initialized: now 1790551672101732 us; error 56 us; skew 500 ppm
I20260927 23:27:52.102736 2731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:27:52.110160 2731 webserver.cc:460] Webserver started at http://127.2.57.2:40351/ using document root <none> and password file <none>
I20260927 23:27:52.111222 2731 fs_manager.cc:359] Metadata directory not provided
I20260927 23:27:52.111495 2731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:27:52.111997 2731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:27:52.117532 2731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/data/instance:
uuid: "abe182b2afc64d908f692ef45eae1d51"
format_stamp: "Formatted at 2026-09-27 23:27:52 on dist-test-slave-jbx2"
I20260927 23:27:52.118803 2731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/instance:
uuid: "abe182b2afc64d908f692ef45eae1d51"
format_stamp: "Formatted at 2026-09-27 23:27:52 on dist-test-slave-jbx2"
I20260927 23:27:52.127480 2731 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.004s
I20260927 23:27:52.133859 2759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:27:52.135176 2731 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.005s
I20260927 23:27:52.135527 2731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal
uuid: "abe182b2afc64d908f692ef45eae1d51"
format_stamp: "Formatted at 2026-09-27 23:27:52 on dist-test-slave-jbx2"
I20260927 23:27:52.135905 2731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:27:52.174465 2731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:27:52.176194 2731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:27:52.176698 2731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:27:52.179600 2731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:27:52.184262 2731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:27:52.184544 2731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:27:52.184803 2731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:27:52.184962 2731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:27:52.273737 2825 raft_consensus.cc:482] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:27:52.274319 2825 raft_consensus.cc:504] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:52.277778 2825 leader_election.cc:288] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:27:52.277642 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" is_pre_election: true
I20260927 23:27:52.277678 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:27:52.279938 2302 leader_election.cc:302] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bee21c28e51643189178a3c851672546; no voters: 0cdd64d48a684d5ebccc2e2e8a4e3150, 19a88ff0709d4266b933cc526577127f
I20260927 23:27:52.281035 2825 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:27:52.343374 2731 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.2:41573
I20260927 23:27:52.343504 2872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.2:41573 every 8 connection(s)
I20260927 23:27:52.346371 2731 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/data/info.pb
I20260927 23:27:52.351092 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 2731
I20260927 23:27:52.351845 2276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/instance
I20260927 23:27:52.367694 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.57.3:0
--local_ip_for_outbound_sockets=127.2.57.3
--webserver_interface=127.2.57.3
--webserver_port=0
--tserver_master_addrs=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--builtin_ntp_servers=127.2.57.20:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260927 23:27:52.415691 2875 heartbeater.cc:347] Connected to a master server at 127.2.57.62:38773
I20260927 23:27:52.416265 2875 heartbeater.cc:464] Registering TS with master...
I20260927 23:27:52.417708 2875 heartbeater.cc:511] Master 127.2.57.62:38773 requested a full tablet report, sending...
I20260927 23:27:52.419560 2873 heartbeater.cc:347] Connected to a master server at 127.2.57.60:41571
I20260927 23:27:52.419907 2873 heartbeater.cc:464] Registering TS with master...
I20260927 23:27:52.420809 2873 heartbeater.cc:511] Master 127.2.57.60:41571 requested a full tablet report, sending...
I20260927 23:27:52.421870 2318 ts_manager.cc:194] Registered new tserver with Master: abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:52.423646 2472 ts_manager.cc:194] Registered new tserver with Master: abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:52.429402 2874 heartbeater.cc:347] Connected to a master server at 127.2.57.61:34987
I20260927 23:27:52.429803 2874 heartbeater.cc:464] Registering TS with master...
I20260927 23:27:52.430761 2874 heartbeater.cc:511] Master 127.2.57.61:34987 requested a full tablet report, sending...
I20260927 23:27:52.433203 2393 ts_manager.cc:194] Registered new tserver with Master: abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:52.435304 2393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.57.2:55769
I20260927 23:27:52.628620 2884 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:27:52.629153 2884 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:52.630967 2884 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987)
I20260927 23:27:52.632007 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:27:52.632215 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:27:52.633745 2456 leader_election.cc:302] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150; no voters: 19a88ff0709d4266b933cc526577127f, bee21c28e51643189178a3c851672546
I20260927 23:27:52.634459 2884 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:27:52.742657 2883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:27:52.743242 2883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:27:52.743850 2883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:27:52.777956 2883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:27:52.779021 2883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.57.3
I20260927 23:27:52.817766 2883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.57.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.57.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.57.3
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:27:52.819321 2883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:27:52.821364 2883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:27:52.843978 2891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:52.939930 2874 heartbeater.cc:503] Master 127.2.57.61:34987 was elected leader, sending a full tablet report...
I20260927 23:27:53.251528 2897 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:27:53.251997 2897 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:53.253762 2897 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987)
I20260927 23:27:53.255295 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:27:53.256289 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:27:53.258903 2456 leader_election.cc:302] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150; no voters: 19a88ff0709d4266b933cc526577127f, bee21c28e51643189178a3c851672546
I20260927 23:27:53.259778 2897 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:27:53.621635 2898 raft_consensus.cc:482] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:27:53.622217 2898 raft_consensus.cc:504] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:53.624691 2898 leader_election.cc:288] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:27:53.625398 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:27:53.625586 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" is_pre_election: true
I20260927 23:27:53.627182 2302 leader_election.cc:302] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bee21c28e51643189178a3c851672546; no voters: 0cdd64d48a684d5ebccc2e2e8a4e3150, 19a88ff0709d4266b933cc526577127f
I20260927 23:27:53.627948 2898 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:27:52.844208 2890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:52.846346 2883 server_base.cc:1034] running on GCE node
W20260927 23:27:52.845346 2893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:27:53.980059 2883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:27:53.983114 2883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:27:53.984606 2883 hybrid_clock.cc:648] HybridClock initialized: now 1790551673984544 us; error 69 us; skew 500 ppm
I20260927 23:27:53.985653 2883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:27:53.992805 2883 webserver.cc:460] Webserver started at http://127.2.57.3:45919/ using document root <none> and password file <none>
I20260927 23:27:53.993866 2883 fs_manager.cc:359] Metadata directory not provided
I20260927 23:27:53.994091 2883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:27:53.994573 2883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:27:54.000167 2883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/data/instance:
uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
format_stamp: "Formatted at 2026-09-27 23:27:53 on dist-test-slave-jbx2"
I20260927 23:27:54.001534 2883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/instance:
uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
format_stamp: "Formatted at 2026-09-27 23:27:53 on dist-test-slave-jbx2"
I20260927 23:27:54.009801 2883 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260927 23:27:54.016573 2903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:27:54.017953 2883 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:27:54.018338 2883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal
uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
format_stamp: "Formatted at 2026-09-27 23:27:53 on dist-test-slave-jbx2"
I20260927 23:27:54.018757 2883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:27:54.057534 2883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:27:54.059875 2883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:27:54.060516 2883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:27:54.064381 2883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:27:54.073450 2883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:27:54.073699 2883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:27:54.074007 2883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:27:54.074254 2883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260927 23:27:54.161232 2960 raft_consensus.cc:482] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:27:54.161830 2960 raft_consensus.cc:504] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:54.164144 2960 leader_election.cc:288] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:27:54.165299 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:27:54.166803 2482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" is_pre_election: true
I20260927 23:27:54.168545 2303 leader_election.cc:302] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bee21c28e51643189178a3c851672546; no voters: 0cdd64d48a684d5ebccc2e2e8a4e3150, 19a88ff0709d4266b933cc526577127f
I20260927 23:27:54.169319 2960 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:27:54.242375 2883 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.3:33209
I20260927 23:27:54.242576 3016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.3:33209 every 8 connection(s)
I20260927 23:27:54.245424 2883 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/data/info.pb
I20260927 23:27:54.254432 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 2883
I20260927 23:27:54.255024 2276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/instance
I20260927 23:27:54.301010 3017 heartbeater.cc:347] Connected to a master server at 127.2.57.60:41571
I20260927 23:27:54.301640 3017 heartbeater.cc:464] Registering TS with master...
I20260927 23:27:54.303011 3017 heartbeater.cc:511] Master 127.2.57.60:41571 requested a full tablet report, sending...
I20260927 23:27:54.305507 3018 heartbeater.cc:347] Connected to a master server at 127.2.57.61:34987
I20260927 23:27:54.305581 3019 heartbeater.cc:347] Connected to a master server at 127.2.57.62:38773
I20260927 23:27:54.305817 3018 heartbeater.cc:464] Registering TS with master...
I20260927 23:27:54.306020 3019 heartbeater.cc:464] Registering TS with master...
I20260927 23:27:54.306241 2472 ts_manager.cc:194] Registered new tserver with Master: de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:54.306667 3018 heartbeater.cc:511] Master 127.2.57.61:34987 requested a full tablet report, sending...
I20260927 23:27:54.306886 3019 heartbeater.cc:511] Master 127.2.57.62:38773 requested a full tablet report, sending...
I20260927 23:27:54.308748 2393 ts_manager.cc:194] Registered new tserver with Master: de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:54.309237 2318 ts_manager.cc:194] Registered new tserver with Master: de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:54.310537 2393 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.57.3:33427
I20260927 23:27:54.323454 2276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:27:54.377872 3034 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:27:54.378399 3034 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:27:54.380506 3034 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987)
I20260927 23:27:54.381147 2328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:27:54.381742 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:27:54.383381 2456 leader_election.cc:302] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150; no voters: 19a88ff0709d4266b933cc526577127f, bee21c28e51643189178a3c851672546
I20260927 23:27:54.384686 3034 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:27:54.436806 2276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu with pid 2430
I20260927 23:27:54.395263 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_50779a5a010b4351a6a795775b9354c6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:54.406477 2390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_82186b684a1544d7b3cca9242b77688a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:54.400594 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_05c6d455c6cc465f90a963710312a5c3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:54.395355 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_55303f56914b4bb5bb4275709a0e13b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:54.449517 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_55303f56914b4bb5bb4275709a0e13b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:27:54.450814 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_05c6d455c6cc465f90a963710312a5c3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:27:54.449503 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50779a5a010b4351a6a795775b9354c6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:27:54.452368 2390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_82186b684a1544d7b3cca9242b77688a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:27:54.477384 2873 heartbeater.cc:644] Failed to heartbeat to 127.2.57.60:41571 (0 consecutive failures): Network error: Failed to send heartbeat to master: recv got EOF from 127.2.57.60:41571 (error 108)
W20260927 23:27:54.477698 2378 connection.cc:535] client connection to 127.2.57.60:41571 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:27:54.478092 2378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 6 similar messages]
I20260927 23:27:54.508095 2392 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_50779a5a010b4351a6a795775b9354c6" } new_table_name: "default.Table_624fb0c585614c3db613a9d50d69642d" modify_external_catalogs: true
I20260927 23:27:54.513070 2391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_55303f56914b4bb5bb4275709a0e13b8" } new_table_name: "default.Table_07b48e8af383404c80196f58f95ba9c2" modify_external_catalogs: true
W20260927 23:27:54.517079 2378 consensus_peers.cc:487] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f -> Peer 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571): Couldn't send request to peer 0cdd64d48a684d5ebccc2e2e8a4e3150. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 23:27:54.523027 2390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_05c6d455c6cc465f90a963710312a5c3" } modify_external_catalogs: true
I20260927 23:27:54.523875 2390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_05c6d455c6cc465f90a963710312a5c3" } modify_external_catalogs: true
I20260927 23:27:54.524515 2393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_82186b684a1544d7b3cca9242b77688a" } modify_external_catalogs: true
I20260927 23:27:54.525087 2393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_82186b684a1544d7b3cca9242b77688a" } modify_external_catalogs: true
W20260927 23:27:54.585319 2391 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet beaef767461647149eed1752cb18932c has no consensus state
I20260927 23:27:54.587718 2393 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet c47af72211cc4a1db9d4d141700931df
W20260927 23:27:54.587667 2392 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet af3004c09be14ebdbee12ce87c95e993 consensus state has no leader
I20260927 23:27:54.599131 2654 tablet_service.cc:1467] Processing CreateTablet for tablet af3004c09be14ebdbee12ce87c95e993 (DEFAULT_TABLE table=default.Table_50779a5a010b4351a6a795775b9354c6 [id=c1a377dfa52a4e62b53dc8ffbe3ae6dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:54.603978 2951 tablet_service.cc:1467] Processing CreateTablet for tablet af3004c09be14ebdbee12ce87c95e993 (DEFAULT_TABLE table=default.Table_50779a5a010b4351a6a795775b9354c6 [id=c1a377dfa52a4e62b53dc8ffbe3ae6dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:54.605894 2654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af3004c09be14ebdbee12ce87c95e993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:54.607142 2951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af3004c09be14ebdbee12ce87c95e993. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:27:54.615947 2721 heartbeater.cc:644] Failed to heartbeat to 127.2.57.60:41571 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
W20260927 23:27:54.617113 2380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet beaef767461647149eed1752cb18932c (table default.Table_07b48e8af383404c80196f58f95ba9c2 [id=8ffc72b15ec6464ba1e10be014af68ed], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet beaef767461647149eed1752cb18932c has no consensus state
I20260927 23:27:54.630007 2390 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 735518516ad840dd8afce85263c76f3a
I20260927 23:27:54.642592 2807 tablet_service.cc:1467] Processing CreateTablet for tablet af3004c09be14ebdbee12ce87c95e993 (DEFAULT_TABLE table=default.Table_50779a5a010b4351a6a795775b9354c6 [id=c1a377dfa52a4e62b53dc8ffbe3ae6dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:54.646154 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af3004c09be14ebdbee12ce87c95e993. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:54.648865 3070 tablet_bootstrap.cc:492] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:54.658620 2806 tablet_service.cc:1467] Processing CreateTablet for tablet beaef767461647149eed1752cb18932c (DEFAULT_TABLE table=default.Table_07b48e8af383404c80196f58f95ba9c2 [id=8ffc72b15ec6464ba1e10be014af68ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:54.659529 2950 tablet_service.cc:1467] Processing CreateTablet for tablet beaef767461647149eed1752cb18932c (DEFAULT_TABLE table=default.Table_07b48e8af383404c80196f58f95ba9c2 [id=8ffc72b15ec6464ba1e10be014af68ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:54.660892 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beaef767461647149eed1752cb18932c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:54.660180 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beaef767461647149eed1752cb18932c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:54.663583 2654 tablet_service.cc:1467] Processing CreateTablet for tablet beaef767461647149eed1752cb18932c (DEFAULT_TABLE table=default.Table_07b48e8af383404c80196f58f95ba9c2 [id=8ffc72b15ec6464ba1e10be014af68ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:54.665851 2654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet beaef767461647149eed1752cb18932c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:54.685102 3072 tablet_bootstrap.cc:492] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:54.692800 3072 tablet_bootstrap.cc:654] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:54.695403 3072 log.cc:824] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: Log is configured to *not* fsync() on all Append() calls
I20260927 23:27:54.696942 3070 tablet_bootstrap.cc:654] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:54.700030 3070 log.cc:824] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: Log is configured to *not* fsync() on all Append() calls
I20260927 23:27:54.705900 2390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_972bb59499344429b243cac7b8f09e55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:54.707770 2390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_972bb59499344429b243cac7b8f09e55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:54.711669 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_f296443ec711403c81c40ea44bc5fb36"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:54.713176 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f296443ec711403c81c40ea44bc5fb36 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:54.715246 3072 tablet_bootstrap.cc:492] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:54.715790 3075 tablet_bootstrap.cc:492] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:54.715859 3072 ts_tablet_manager.cc:1397] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.031s user 0.016s sys 0.013s
I20260927 23:27:54.722087 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_719182fbf114487d99cfa2e35fd9531e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:54.726359 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_9e8530d40a5d49efae5ff9e54846bb86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:54.728438 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e8530d40a5d49efae5ff9e54846bb86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:54.740061 3070 tablet_bootstrap.cc:492] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:54.740718 3070 ts_tablet_manager.cc:1397] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.093s user 0.012s sys 0.009s
I20260927 23:27:54.743264 3075 tablet_bootstrap.cc:654] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:54.745944 3075 log.cc:824] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: Log is configured to *not* fsync() on all Append() calls
I20260927 23:27:54.757623 3072 raft_consensus.cc:348] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:54.758841 3072 raft_consensus.cc:374] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:54.759150 3072 raft_consensus.cc:729] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:54.760322 3072 consensus_queue.cc:260] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:27:54.761993 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_719182fbf114487d99cfa2e35fd9531e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:54.762378 3075 tablet_bootstrap.cc:492] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:54.763043 3075 ts_tablet_manager.cc:1397] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.048s user 0.015s sys 0.000s
I20260927 23:27:54.771051 3072 ts_tablet_manager.cc:1428] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.055s user 0.032s sys 0.011s
W20260927 23:27:54.776459 3017 heartbeater.cc:644] Failed to heartbeat to 127.2.57.60:41571 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
I20260927 23:27:54.779700 3018 heartbeater.cc:503] Master 127.2.57.61:34987 was elected leader, sending a full tablet report...
I20260927 23:27:54.783974 3072 tablet_bootstrap.cc:492] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:54.787040 2654 tablet_service.cc:1467] Processing CreateTablet for tablet 4f42104ac713431e84cad2ac0b204b4e (DEFAULT_TABLE table=default.Table_972bb59499344429b243cac7b8f09e55 [id=7032872e5f73441caff3a267804f716d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:54.788270 2654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f42104ac713431e84cad2ac0b204b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:54.788190 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 4f42104ac713431e84cad2ac0b204b4e (DEFAULT_TABLE table=default.Table_972bb59499344429b243cac7b8f09e55 [id=7032872e5f73441caff3a267804f716d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:54.789322 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f42104ac713431e84cad2ac0b204b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:54.793831 2807 tablet_service.cc:1467] Processing CreateTablet for tablet 4f42104ac713431e84cad2ac0b204b4e (DEFAULT_TABLE table=default.Table_972bb59499344429b243cac7b8f09e55 [id=7032872e5f73441caff3a267804f716d]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:27:54.794994 3024 tablet.cc:2369] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:27:54.795167 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f42104ac713431e84cad2ac0b204b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:54.809608 2389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_f296443ec711403c81c40ea44bc5fb36" } new_table_name: "default.Table_66cf99f0a24e4ae5911400243ed6ebbd" modify_external_catalogs: true
I20260927 23:27:54.813854 2390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_972bb59499344429b243cac7b8f09e55" } modify_external_catalogs: true
I20260927 23:27:54.814476 2390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_972bb59499344429b243cac7b8f09e55" } modify_external_catalogs: true
I20260927 23:27:54.816745 2388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_624fb0c585614c3db613a9d50d69642d" } new_table_name: "default.Table_bc2a5a98bf2b443d999a2ecaa81e6309" modify_external_catalogs: true
I20260927 23:27:54.818662 3070 raft_consensus.cc:348] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:54.810393 3072 tablet_bootstrap.cc:654] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:54.831581 3072 tablet_bootstrap.cc:492] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:54.832052 3072 ts_tablet_manager.cc:1397] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.048s user 0.007s sys 0.004s
I20260927 23:27:54.834587 2387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_07b48e8af383404c80196f58f95ba9c2" } modify_external_catalogs: true
I20260927 23:27:54.835093 3072 raft_consensus.cc:348] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:54.835459 2387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_07b48e8af383404c80196f58f95ba9c2" } modify_external_catalogs: true
I20260927 23:27:54.835768 3072 raft_consensus.cc:374] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:54.836032 3072 raft_consensus.cc:729] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:54.836874 3072 consensus_queue.cc:260] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:54.838208 3070 raft_consensus.cc:374] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:54.844594 3070 raft_consensus.cc:729] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:54.844847 3072 ts_tablet_manager.cc:1428] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260927 23:27:54.846028 3070 consensus_queue.cc:260] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:54.848095 3072 tablet_bootstrap.cc:492] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:54.855384 3072 tablet_bootstrap.cc:654] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:54.862787 3070 ts_tablet_manager.cc:1428] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.122s user 0.039s sys 0.001s
I20260927 23:27:54.864073 3070 tablet_bootstrap.cc:492] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:54.880614 3077 raft_consensus.cc:482] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:54.881274 3077 raft_consensus.cc:504] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:54.889798 3077 leader_election.cc:288] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:54.901371 3070 tablet_bootstrap.cc:654] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:54.903990 3075 raft_consensus.cc:348] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:54.907544 3072 tablet_bootstrap.cc:492] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:54.908084 3072 ts_tablet_manager.cc:1397] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.060s user 0.011s sys 0.025s
I20260927 23:27:54.908473 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beaef767461647149eed1752cb18932c" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:27:54.912400 2389 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d6cfa2547e6a497a8db35651b03b9995 has no consensus state
W20260927 23:27:54.913453 2907 leader_election.cc:341] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:54.910756 3072 raft_consensus.cc:348] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:54.915445 3072 raft_consensus.cc:374] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:54.915752 3072 raft_consensus.cc:729] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:54.916448 3072 consensus_queue.cc:260] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:54.918576 3072 ts_tablet_manager.cc:1428] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260927 23:27:54.919296 3075 raft_consensus.cc:374] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:54.919675 3075 raft_consensus.cc:729] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:54.921401 3075 consensus_queue.cc:260] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:54.935295 3075 ts_tablet_manager.cc:1428] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.172s user 0.034s sys 0.008s
I20260927 23:27:54.936381 3075 tablet_bootstrap.cc:492] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:54.938841 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "beaef767461647149eed1752cb18932c" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:54.939549 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_8166baf5795042f5b79dd39a7d545027"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:54.941453 2906 leader_election.cc:341] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:54.941983 2906 leader_election.cc:302] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:54.943552 3077 raft_consensus.cc:2726] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:27:54.950163 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8166baf5795042f5b79dd39a7d545027 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:54.952929 3075 tablet_bootstrap.cc:654] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:54.968753 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_e3fad1f8c2494735b4723d0ecc1b0b39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:54.971441 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e3fad1f8c2494735b4723d0ecc1b0b39 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:54.979802 2654 tablet_service.cc:1467] Processing CreateTablet for tablet d6cfa2547e6a497a8db35651b03b9995 (DEFAULT_TABLE table=default.Table_66cf99f0a24e4ae5911400243ed6ebbd [id=18c041f9985b4c9f99cbc8cc4f74db94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:54.981077 2654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6cfa2547e6a497a8db35651b03b9995. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:27:54.981262 2728 tablet.cc:2369] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:27:54.981557 2806 tablet_service.cc:1467] Processing CreateTablet for tablet d6cfa2547e6a497a8db35651b03b9995 (DEFAULT_TABLE table=default.Table_66cf99f0a24e4ae5911400243ed6ebbd [id=18c041f9985b4c9f99cbc8cc4f74db94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:54.982615 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6cfa2547e6a497a8db35651b03b9995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:54.985791 2950 tablet_service.cc:1467] Processing CreateTablet for tablet d6cfa2547e6a497a8db35651b03b9995 (DEFAULT_TABLE table=default.Table_66cf99f0a24e4ae5911400243ed6ebbd [id=18c041f9985b4c9f99cbc8cc4f74db94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:54.987046 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6cfa2547e6a497a8db35651b03b9995. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.002187 3070 tablet_bootstrap.cc:492] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:55.002827 3072 tablet_bootstrap.cc:492] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:55.005193 3070 ts_tablet_manager.cc:1397] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.141s user 0.015s sys 0.018s
I20260927 23:27:55.009447 3072 tablet_bootstrap.cc:654] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.011376 3070 raft_consensus.cc:348] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.026072 3072 tablet_bootstrap.cc:492] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:55.026611 3072 ts_tablet_manager.cc:1397] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.002s
I20260927 23:27:55.029654 3072 raft_consensus.cc:348] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.030535 3072 raft_consensus.cc:374] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.030803 3072 raft_consensus.cc:729] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.036831 3072 consensus_queue.cc:260] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.043043 3072 ts_tablet_manager.cc:1428] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.016s user 0.006s sys 0.002s
I20260927 23:27:55.064568 3070 raft_consensus.cc:374] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.065003 3070 raft_consensus.cc:729] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.066064 3070 consensus_queue.cc:260] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.071691 2390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 4f42104ac713431e84cad2ac0b204b4e
I20260927 23:27:55.072206 3070 ts_tablet_manager.cc:1428] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.064s user 0.004s sys 0.002s
I20260927 23:27:55.073462 3070 tablet_bootstrap.cc:492] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:55.076521 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f42104ac713431e84cad2ac0b204b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:54 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:55.077168 2654 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f42104ac713431e84cad2ac0b204b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:54 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:55.083472 2392 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet af3004c09be14ebdbee12ce87c95e993 (table default.Table_624fb0c585614c3db613a9d50d69642d [id=c1a377dfa52a4e62b53dc8ffbe3ae6dc]). Expected version 1 got 0
I20260927 23:27:55.084105 3075 tablet_bootstrap.cc:492] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:55.084702 3075 ts_tablet_manager.cc:1397] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.149s user 0.040s sys 0.042s
I20260927 23:27:55.087086 2386 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet af3004c09be14ebdbee12ce87c95e993 (table default.Table_624fb0c585614c3db613a9d50d69642d [id=c1a377dfa52a4e62b53dc8ffbe3ae6dc]). Expected version 1 got 0
I20260927 23:27:55.087775 3075 raft_consensus.cc:348] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.088660 3075 raft_consensus.cc:374] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.088963 3075 raft_consensus.cc:729] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.089468 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet af3004c09be14ebdbee12ce87c95e993 (table default.Table_624fb0c585614c3db613a9d50d69642d [id=c1a377dfa52a4e62b53dc8ffbe3ae6dc]). Expected version 1 got 0
I20260927 23:27:55.089828 3075 consensus_queue.cc:260] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:27:55.091188 2388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet af3004c09be14ebdbee12ce87c95e993 consensus state has no leader
W20260927 23:27:55.091885 2424 debug-util.cc:398] Leaking SignalData structure 0x7b08000aae40 after lost signal to thread 2359
I20260927 23:27:55.093115 2387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet beaef767461647149eed1752cb18932c
I20260927 23:27:55.096104 3075 ts_tablet_manager.cc:1428] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260927 23:27:55.096758 2951 tablet_service.cc:1514] Processing DeleteTablet for tablet beaef767461647149eed1752cb18932c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:54 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:55.101083 3075 tablet_bootstrap.cc:492] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:55.107081 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f42104ac713431e84cad2ac0b204b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:54 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:55.110656 2653 tablet_service.cc:1514] Processing DeleteTablet for tablet beaef767461647149eed1752cb18932c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:54 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:55.111754 3070 tablet_bootstrap.cc:654] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.111959 3075 tablet_bootstrap.cc:654] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.113354 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet beaef767461647149eed1752cb18932c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:54 UTC) from {username='slave'} at 127.0.0.1:42492
W20260927 23:27:55.116981 2880 tablet.cc:2369] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:27:55.133641 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_b37cbe1669c84445b5020abfa31fe644"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:55.135464 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b37cbe1669c84445b5020abfa31fe644 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:55.137679 2389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_719182fbf114487d99cfa2e35fd9531e" } new_table_name: "default.Table_1b6c64cbb7304c48a43dfa5cef404376" modify_external_catalogs: true
I20260927 23:27:55.146399 3070 tablet_bootstrap.cc:492] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:55.146961 3070 ts_tablet_manager.cc:1397] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.074s user 0.013s sys 0.005s
I20260927 23:27:55.157709 3075 tablet_bootstrap.cc:492] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:55.158514 3075 ts_tablet_manager.cc:1397] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.058s user 0.006s sys 0.010s
I20260927 23:27:55.159091 3077 raft_consensus.cc:482] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:55.157918 3070 raft_consensus.cc:348] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.159677 3077 raft_consensus.cc:504] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.160342 3070 raft_consensus.cc:374] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.160645 3070 raft_consensus.cc:729] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.161326 3070 consensus_queue.cc:260] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.163897 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cfa2547e6a497a8db35651b03b9995" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:55.161782 3075 raft_consensus.cc:348] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.165221 3075 raft_consensus.cc:374] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.165120 3077 leader_election.cc:288] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:55.165584 3075 raft_consensus.cc:729] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
W20260927 23:27:55.166443 2906 leader_election.cc:341] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:55.168502 3070 ts_tablet_manager.cc:1428] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.021s user 0.009s sys 0.000s
I20260927 23:27:55.169474 3070 tablet_bootstrap.cc:492] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:55.170435 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cfa2547e6a497a8db35651b03b9995" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:55.166415 3075 consensus_queue.cc:260] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:27:55.171159 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 4f42104ac713431e84cad2ac0b204b4e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4f42104ac713431e84cad2ac0b204b4e already in progress: creating tablet
W20260927 23:27:55.173007 2907 leader_election.cc:341] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:55.173454 2907 leader_election.cc:302] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:55.174763 3077 raft_consensus.cc:2726] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:55.175405 3070 tablet_bootstrap.cc:654] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.132241 3053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "19a88ff0709d4266b933cc526577127f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:27:55.177053 3053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [sys.catalog]: This master's current role is: LEADER
I20260927 23:27:55.177963 3092 tablet_replica.cc:330] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:27:55.179044 3075 ts_tablet_manager.cc:1428] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.020s user 0.004s sys 0.002s
I20260927 23:27:55.172989 3090 tablet_replica.cc:330] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:27:55.184357 3075 tablet_bootstrap.cc:492] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:55.185357 3090 raft_consensus.cc:2229] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:55.186009 3090 raft_consensus.cc:2258] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:55.187762 3091 tablet_replica.cc:330] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:27:55.188768 3091 raft_consensus.cc:2229] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:55.189393 3091 raft_consensus.cc:2258] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:55.184777 3092 raft_consensus.cc:2229] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:55.190443 3092 raft_consensus.cc:2258] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:55.194416 3070 tablet_bootstrap.cc:492] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:55.194908 3070 ts_tablet_manager.cc:1397] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.026s user 0.013s sys 0.003s
I20260927 23:27:55.196638 3092 ts_tablet_manager.cc:1905] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:55.197562 3070 raft_consensus.cc:348] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.198299 3070 raft_consensus.cc:374] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.198582 3070 raft_consensus.cc:729] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.199414 3070 consensus_queue.cc:260] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.198074 3075 tablet_bootstrap.cc:654] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.201906 3070 ts_tablet_manager.cc:1428] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260927 23:27:55.203920 3090 ts_tablet_manager.cc:1905] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:55.202956 3091 ts_tablet_manager.cc:1905] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:55.231724 3090 ts_tablet_manager.cc:1918] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:55.234917 2391 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet d6cfa2547e6a497a8db35651b03b9995 (table default.Table_66cf99f0a24e4ae5911400243ed6ebbd [id=18c041f9985b4c9f99cbc8cc4f74db94]). Expected version 1 got 0
I20260927 23:27:55.235955 2654 tablet_service.cc:1467] Processing CreateTablet for tablet 502bdd29d2ab42579ae0239a7b6679c6 (DEFAULT_TABLE table=default.Table_9e8530d40a5d49efae5ff9e54846bb86 [id=7895554af47d4b439fe421f68bb5a9a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.237272 2654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 502bdd29d2ab42579ae0239a7b6679c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.237645 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 502bdd29d2ab42579ae0239a7b6679c6 (DEFAULT_TABLE table=default.Table_9e8530d40a5d49efae5ff9e54846bb86 [id=7895554af47d4b439fe421f68bb5a9a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.237123 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 502bdd29d2ab42579ae0239a7b6679c6 (DEFAULT_TABLE table=default.Table_9e8530d40a5d49efae5ff9e54846bb86 [id=7895554af47d4b439fe421f68bb5a9a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.238794 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 502bdd29d2ab42579ae0239a7b6679c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.239168 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 502bdd29d2ab42579ae0239a7b6679c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.239456 2387 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet af3004c09be14ebdbee12ce87c95e993 (table default.Table_bc2a5a98bf2b443d999a2ecaa81e6309 [id=c1a377dfa52a4e62b53dc8ffbe3ae6dc]). Expected version 2 got 0
W20260927 23:27:55.251698 2389 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a4372ae7a9d6426a97419d89253f99d1 has no consensus state
I20260927 23:27:55.252979 3090 log.cc:1196] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/4f42104ac713431e84cad2ac0b204b4e
I20260927 23:27:55.254705 3075 tablet_bootstrap.cc:492] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:55.255298 3075 ts_tablet_manager.cc:1397] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.071s user 0.008s sys 0.007s
I20260927 23:27:55.255991 3090 ts_tablet_manager.cc:1939] T 4f42104ac713431e84cad2ac0b204b4e P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:27:55.258217 3075 raft_consensus.cc:348] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.258991 3075 raft_consensus.cc:374] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.259251 3075 raft_consensus.cc:729] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.278514 3075 consensus_queue.cc:260] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.280983 3070 tablet_bootstrap.cc:492] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:55.282793 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 4f42104ac713431e84cad2ac0b204b4e (table default.Table_972bb59499344429b243cac7b8f09e55 [id=7032872e5f73441caff3a267804f716d]) successfully deleted
I20260927 23:27:55.283243 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_73c1ac5a47a045529546f8ff56205202"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:55.284808 3072 tablet_bootstrap.cc:492] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
W20260927 23:27:55.285279 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73c1ac5a47a045529546f8ff56205202 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:55.291445 3070 tablet_bootstrap.cc:654] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.291829 2389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_9e8530d40a5d49efae5ff9e54846bb86" } modify_external_catalogs: true
I20260927 23:27:55.292829 2389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_9e8530d40a5d49efae5ff9e54846bb86" } modify_external_catalogs: true
I20260927 23:27:55.293107 3075 ts_tablet_manager.cc:1428] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.038s user 0.007s sys 0.002s
I20260927 23:27:55.295125 3072 tablet_bootstrap.cc:654] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.295401 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f42104ac713431e84cad2ac0b204b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:54 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:55.296700 3090 tablet_replica.cc:330] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:27:55.297492 3090 raft_consensus.cc:2229] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:55.297544 3084 raft_consensus.cc:482] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:55.297998 3090 raft_consensus.cc:2258] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:55.298090 3084 raft_consensus.cc:504] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.302570 2388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_66cf99f0a24e4ae5911400243ed6ebbd" } new_table_name: "default.Table_0b552fe90a6342eeb8c086f4c7436f07" modify_external_catalogs: true
I20260927 23:27:55.306581 2387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_e3fad1f8c2494735b4723d0ecc1b0b39" } modify_external_catalogs: true
I20260927 23:27:55.307139 2387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_e3fad1f8c2494735b4723d0ecc1b0b39" } modify_external_catalogs: true
I20260927 23:27:55.316740 3084 leader_election.cc:288] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:55.319321 3090 ts_tablet_manager.cc:1905] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:55.326787 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f42104ac713431e84cad2ac0b204b4e" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:27:55.328621 2763 leader_election.cc:341] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: Tablet not RUNNING: STOPPED
I20260927 23:27:55.336114 3090 ts_tablet_manager.cc:1918] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:55.337404 3075 tablet_bootstrap.cc:492] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:55.343341 3070 tablet_bootstrap.cc:492] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:55.343838 3070 ts_tablet_manager.cc:1397] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.063s user 0.007s sys 0.005s
I20260927 23:27:55.346556 3070 raft_consensus.cc:348] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:55.347344 3070 raft_consensus.cc:374] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.347664 3070 raft_consensus.cc:729] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.350272 3091 ts_tablet_manager.cc:1918] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:55.356287 2385 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet d6cfa2547e6a497a8db35651b03b9995 (table default.Table_66cf99f0a24e4ae5911400243ed6ebbd [id=18c041f9985b4c9f99cbc8cc4f74db94]). Expected version 1 got 0
I20260927 23:27:55.356077 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_b316d4f7e203499bb394d7c026fe761a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:55.361035 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b316d4f7e203499bb394d7c026fe761a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:55.361131 2386 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet d6cfa2547e6a497a8db35651b03b9995 (table default.Table_66cf99f0a24e4ae5911400243ed6ebbd [id=18c041f9985b4c9f99cbc8cc4f74db94]). Expected version 1 got 0
W20260927 23:27:55.362470 2388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d6cfa2547e6a497a8db35651b03b9995 consensus state has no leader
I20260927 23:27:55.363476 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_9fcb69a219404bf4a763d08ffb0a744a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:55.365172 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9fcb69a219404bf4a763d08ffb0a744a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:55.371425 2807 tablet_service.cc:1467] Processing CreateTablet for tablet 517d7bfc225a4372bfd79682a550a0d1 (DEFAULT_TABLE table=default.Table_8166baf5795042f5b79dd39a7d545027 [id=4a83aea286194e6fac09fde39f326c9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.372718 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 517d7bfc225a4372bfd79682a550a0d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.376746 2389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 502bdd29d2ab42579ae0239a7b6679c6
I20260927 23:27:55.378589 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 26d4c74dddca44bdbaac368e8d521fe7 (DEFAULT_TABLE table=default.Table_b37cbe1669c84445b5020abfa31fe644 [id=dd99f8473f9a4f4f82b2127c1ea61fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.378777 2805 tablet_service.cc:1467] Processing CreateTablet for tablet a4372ae7a9d6426a97419d89253f99d1 (DEFAULT_TABLE table=default.Table_1b6c64cbb7304c48a43dfa5cef404376 [id=5f884e6c6e6445bbbbdeacc26cd347a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.379716 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26d4c74dddca44bdbaac368e8d521fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.381145 2804 tablet_service.cc:1467] Processing CreateTablet for tablet b5efb2e4760a48c98e1c6e9c1add0eca (DEFAULT_TABLE table=default.Table_e3fad1f8c2494735b4723d0ecc1b0b39 [id=601abf06edf443879124d16259163a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.382282 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5efb2e4760a48c98e1c6e9c1add0eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.384869 2803 tablet_service.cc:1514] Processing DeleteTablet for tablet 502bdd29d2ab42579ae0239a7b6679c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:55.389066 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4372ae7a9d6426a97419d89253f99d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.394573 3092 ts_tablet_manager.cc:1918] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:55.395022 3092 log.cc:1196] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/beaef767461647149eed1752cb18932c
I20260927 23:27:55.395978 3092 ts_tablet_manager.cc:1939] T beaef767461647149eed1752cb18932c P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:27:55.363353 2654 tablet_service.cc:1467] Processing CreateTablet for tablet 26d4c74dddca44bdbaac368e8d521fe7 (DEFAULT_TABLE table=default.Table_b37cbe1669c84445b5020abfa31fe644 [id=dd99f8473f9a4f4f82b2127c1ea61fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.399518 2654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26d4c74dddca44bdbaac368e8d521fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.384686 3070 consensus_queue.cc:260] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:55.401844 3070 ts_tablet_manager.cc:1428] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.058s user 0.008s sys 0.000s
I20260927 23:27:55.385746 2653 tablet_service.cc:1467] Processing CreateTablet for tablet 517d7bfc225a4372bfd79682a550a0d1 (DEFAULT_TABLE table=default.Table_8166baf5795042f5b79dd39a7d545027 [id=4a83aea286194e6fac09fde39f326c9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.407650 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 517d7bfc225a4372bfd79682a550a0d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.336591 3090 log.cc:1196] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/beaef767461647149eed1752cb18932c
I20260927 23:27:55.410266 3090 ts_tablet_manager.cc:1939] T beaef767461647149eed1752cb18932c P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:27:55.393144 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 26d4c74dddca44bdbaac368e8d521fe7 (DEFAULT_TABLE table=default.Table_b37cbe1669c84445b5020abfa31fe644 [id=dd99f8473f9a4f4f82b2127c1ea61fd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.397276 2951 tablet_service.cc:1467] Processing CreateTablet for tablet 517d7bfc225a4372bfd79682a550a0d1 (DEFAULT_TABLE table=default.Table_8166baf5795042f5b79dd39a7d545027 [id=4a83aea286194e6fac09fde39f326c9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.387945 2652 tablet_service.cc:1467] Processing CreateTablet for tablet a4372ae7a9d6426a97419d89253f99d1 (DEFAULT_TABLE table=default.Table_1b6c64cbb7304c48a43dfa5cef404376 [id=5f884e6c6e6445bbbbdeacc26cd347a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.398712 2949 tablet_service.cc:1467] Processing CreateTablet for tablet a4372ae7a9d6426a97419d89253f99d1 (DEFAULT_TABLE table=default.Table_1b6c64cbb7304c48a43dfa5cef404376 [id=5f884e6c6e6445bbbbdeacc26cd347a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.415686 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4372ae7a9d6426a97419d89253f99d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.400084 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet 502bdd29d2ab42579ae0239a7b6679c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:55.367360 3072 tablet_bootstrap.cc:492] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:55.421195 3072 ts_tablet_manager.cc:1397] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.137s user 0.011s sys 0.002s
I20260927 23:27:55.423224 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26d4c74dddca44bdbaac368e8d521fe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.391294 2651 tablet_service.cc:1467] Processing CreateTablet for tablet b5efb2e4760a48c98e1c6e9c1add0eca (DEFAULT_TABLE table=default.Table_e3fad1f8c2494735b4723d0ecc1b0b39 [id=601abf06edf443879124d16259163a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.424713 2951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 517d7bfc225a4372bfd79682a550a0d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.399679 2948 tablet_service.cc:1467] Processing CreateTablet for tablet b5efb2e4760a48c98e1c6e9c1add0eca (DEFAULT_TABLE table=default.Table_e3fad1f8c2494735b4723d0ecc1b0b39 [id=601abf06edf443879124d16259163a7e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.428160 3075 tablet_bootstrap.cc:654] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.428699 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5efb2e4760a48c98e1c6e9c1add0eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.429530 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet beaef767461647149eed1752cb18932c (table default.Table_07b48e8af383404c80196f58f95ba9c2 [id=8ffc72b15ec6464ba1e10be014af68ed]) successfully deleted
W20260927 23:27:55.430637 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 502bdd29d2ab42579ae0239a7b6679c6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 502bdd29d2ab42579ae0239a7b6679c6 already in progress: creating tablet
I20260927 23:27:55.432592 2949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4372ae7a9d6426a97419d89253f99d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.423722 3072 raft_consensus.cc:348] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:55.436722 3072 raft_consensus.cc:374] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.437021 3072 raft_consensus.cc:729] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.437800 3072 consensus_queue.cc:260] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:55.445267 3072 ts_tablet_manager.cc:1428] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.024s user 0.005s sys 0.001s
I20260927 23:27:55.392119 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet 502bdd29d2ab42579ae0239a7b6679c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:55.457502 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet beaef767461647149eed1752cb18932c (table default.Table_07b48e8af383404c80196f58f95ba9c2 [id=8ffc72b15ec6464ba1e10be014af68ed]) successfully deleted
I20260927 23:27:55.459095 3092 tablet_replica.cc:330] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:27:55.460680 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5efb2e4760a48c98e1c6e9c1add0eca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.463735 3079 raft_consensus.cc:482] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:55.465286 2389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_b37cbe1669c84445b5020abfa31fe644" } new_table_name: "default.Table_b906f759cf284e0aaf25a13094802a09" modify_external_catalogs: true
I20260927 23:27:55.464366 3079 raft_consensus.cc:504] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:55.461530 3091 log.cc:1196] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/4f42104ac713431e84cad2ac0b204b4e
I20260927 23:27:55.469508 2387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet b5efb2e4760a48c98e1c6e9c1add0eca
I20260927 23:27:55.472482 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_5f321d8241ab4a69a82e325e59605ab2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:55.474334 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f321d8241ab4a69a82e325e59605ab2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:55.479389 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet b5efb2e4760a48c98e1c6e9c1add0eca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:55.479703 3091 ts_tablet_manager.cc:1939] T 4f42104ac713431e84cad2ac0b204b4e P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:27:55.485045 3077 raft_consensus.cc:482] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:55.485595 3077 raft_consensus.cc:504] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:55.493003 3092 raft_consensus.cc:2229] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:55.493593 3092 raft_consensus.cc:2258] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:55.468858 3079 leader_election.cc:288] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:55.503036 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet b5efb2e4760a48c98e1c6e9c1add0eca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:55.514493 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet 502bdd29d2ab42579ae0239a7b6679c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:55.515303 3090 tablet_replica.cc:330] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:27:55.515676 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet b5efb2e4760a48c98e1c6e9c1add0eca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:55.516084 3090 raft_consensus.cc:2229] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:55.516625 3090 raft_consensus.cc:2258] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:55.517632 3092 ts_tablet_manager.cc:1905] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:55.537304 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_f4808e8ee6d64bbb8e824ed11976279d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:55.538581 3091 tablet_replica.cc:330] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
W20260927 23:27:55.538482 2761 leader_election.cc:341] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: 4f42104ac713431e84cad2ac0b204b4e
I20260927 23:27:55.539108 2761 leader_election.cc:302] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:55.539388 3091 raft_consensus.cc:2229] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:55.539846 3091 raft_consensus.cc:2258] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260927 23:27:55.540073 2761 raft_consensus.cc:2670] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260927 23:27:55.540104 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 4f42104ac713431e84cad2ac0b204b4e (table default.Table_972bb59499344429b243cac7b8f09e55 [id=7032872e5f73441caff3a267804f716d]) successfully deleted
W20260927 23:27:55.542676 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f4808e8ee6d64bbb8e824ed11976279d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:55.542996 3091 ts_tablet_manager.cc:1905] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:55.544065 3072 tablet_bootstrap.cc:492] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
W20260927 23:27:55.546013 2389 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 26d4c74dddca44bdbaac368e8d521fe7 consensus state has no leader
I20260927 23:27:55.547742 3075 tablet_bootstrap.cc:492] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:55.548281 3075 ts_tablet_manager.cc:1397] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.211s user 0.014s sys 0.034s
I20260927 23:27:55.550571 3077 leader_election.cc:288] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:55.551307 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502bdd29d2ab42579ae0239a7b6679c6" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:55.553193 3090 ts_tablet_manager.cc:1905] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:55.551154 3075 raft_consensus.cc:348] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:55.565083 3075 raft_consensus.cc:374] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.565475 3075 raft_consensus.cc:729] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.566264 3075 consensus_queue.cc:260] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:55.567556 2828 raft_consensus.cc:2454] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:27:55.568893 3075 ts_tablet_manager.cc:1428] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.020s user 0.001s sys 0.005s
I20260927 23:27:55.571331 2906 leader_election.cc:302] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
W20260927 23:27:55.571966 2906 raft_consensus.cc:2670] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260927 23:27:55.575121 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "502bdd29d2ab42579ae0239a7b6679c6" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:55.575948 2674 raft_consensus.cc:2454] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:27:55.579026 3075 tablet_bootstrap.cc:492] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:55.585489 3075 tablet_bootstrap.cc:654] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.596542 2388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_73c1ac5a47a045529546f8ff56205202" } modify_external_catalogs: true
I20260927 23:27:55.597301 3075 tablet_bootstrap.cc:492] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:55.597283 2388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_73c1ac5a47a045529546f8ff56205202" } modify_external_catalogs: true
I20260927 23:27:55.597761 3075 ts_tablet_manager.cc:1397] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.009s
I20260927 23:27:55.603582 3077 raft_consensus.cc:482] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:55.604118 3077 raft_consensus.cc:504] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:55.612856 3075 raft_consensus.cc:348] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.612991 2389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_b316d4f7e203499bb394d7c026fe761a" } new_table_name: "default.Table_2d889ed8e18e47a79f0541db93242386" modify_external_catalogs: true
I20260927 23:27:55.613584 3075 raft_consensus.cc:374] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.613860 3075 raft_consensus.cc:729] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.623025 3075 consensus_queue.cc:260] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.627933 3092 ts_tablet_manager.cc:1918] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:55.629976 3092 log.cc:1196] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/4f42104ac713431e84cad2ac0b204b4e
I20260927 23:27:55.630743 3075 ts_tablet_manager.cc:1428] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.033s user 0.007s sys 0.000s
I20260927 23:27:55.631699 3092 ts_tablet_manager.cc:1939] T 4f42104ac713431e84cad2ac0b204b4e P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:27:55.634908 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 4f42104ac713431e84cad2ac0b204b4e (table default.Table_972bb59499344429b243cac7b8f09e55 [id=7032872e5f73441caff3a267804f716d]) successfully deleted
I20260927 23:27:55.638432 3072 tablet_bootstrap.cc:654] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.638614 3092 tablet_replica.cc:330] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:27:55.640276 3092 raft_consensus.cc:2229] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:55.642088 3092 raft_consensus.cc:2258] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:55.649041 3075 tablet_bootstrap.cc:492] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:55.650507 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3004c09be14ebdbee12ce87c95e993" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:55.651185 2828 raft_consensus.cc:2454] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:27:55.652505 2906 leader_election.cc:302] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:27:55.645936 3092 ts_tablet_manager.cc:1905] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:27:55.655084 2379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 26d4c74dddca44bdbaac368e8d521fe7 (table default.Table_b906f759cf284e0aaf25a13094802a09 [id=dd99f8473f9a4f4f82b2127c1ea61fd2], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 26d4c74dddca44bdbaac368e8d521fe7 consensus state has no leader
I20260927 23:27:55.670218 3075 tablet_bootstrap.cc:654] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.672196 3077 leader_election.cc:288] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:55.672879 3077 raft_consensus.cc:2781] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:55.673254 3077 raft_consensus.cc:482] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:55.673623 3077 raft_consensus.cc:3037] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:55.680768 2392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_1b6c64cbb7304c48a43dfa5cef404376" } modify_external_catalogs: true
I20260927 23:27:55.681533 2392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_1b6c64cbb7304c48a43dfa5cef404376" } modify_external_catalogs: true
I20260927 23:27:55.689591 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3004c09be14ebdbee12ce87c95e993" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:55.690312 3075 tablet_bootstrap.cc:492] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:55.690294 2828 raft_consensus.cc:2454] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:55.691823 2609 leader_election.cc:302] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:27:55.693511 3079 raft_consensus.cc:2781] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:55.693879 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 2572268ca47346b0b78abc6a9ca7f908 (DEFAULT_TABLE table=default.Table_b316d4f7e203499bb394d7c026fe761a [id=103d7c5e210c48fdafb8c199eb4cf676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.693995 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 2572268ca47346b0b78abc6a9ca7f908 (DEFAULT_TABLE table=default.Table_b316d4f7e203499bb394d7c026fe761a [id=103d7c5e210c48fdafb8c199eb4cf676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.694463 3075 ts_tablet_manager.cc:1397] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.046s user 0.011s sys 0.001s
I20260927 23:27:55.695145 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2572268ca47346b0b78abc6a9ca7f908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.695979 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_d8ee874db95045efa6c1321ce207e4ea"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:55.697058 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 410294c28c49404e9c547b27111712eb (DEFAULT_TABLE table=default.Table_9fcb69a219404bf4a763d08ffb0a744a [id=b215370f7b594ab28c3e2a34de2a6f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.697557 3084 raft_consensus.cc:482] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:55.697242 3075 raft_consensus.cc:348] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:27:55.697765 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d8ee874db95045efa6c1321ce207e4ea in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:55.697906 3075 raft_consensus.cc:374] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.698122 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 410294c28c49404e9c547b27111712eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.698174 3075 raft_consensus.cc:729] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.697944 3084 raft_consensus.cc:504] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.699841 3084 leader_election.cc:288] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:55.693905 3079 raft_consensus.cc:482] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:55.700800 3079 raft_consensus.cc:3037] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:55.700685 3075 consensus_queue.cc:260] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.705437 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_db0f806a353a4dbdb9d64635383bc826"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:55.705921 2649 tablet_service.cc:1467] Processing CreateTablet for tablet 410294c28c49404e9c547b27111712eb (DEFAULT_TABLE table=default.Table_9fcb69a219404bf4a763d08ffb0a744a [id=b215370f7b594ab28c3e2a34de2a6f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.707769 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "517d7bfc225a4372bfd79682a550a0d1" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:55.713187 2951 tablet_service.cc:1467] Processing CreateTablet for tablet 2572268ca47346b0b78abc6a9ca7f908 (DEFAULT_TABLE table=default.Table_b316d4f7e203499bb394d7c026fe761a [id=103d7c5e210c48fdafb8c199eb4cf676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.713269 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3004c09be14ebdbee12ce87c95e993" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:55.714406 2951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2572268ca47346b0b78abc6a9ca7f908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.717525 2807 tablet_service.cc:1467] Processing CreateTablet for tablet a3a5bae1f0624b16be7940ac9b84c4da (DEFAULT_TABLE table=default.Table_73c1ac5a47a045529546f8ff56205202 [id=5744abf74d4f405d9fc6a7e70804b513]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.718706 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3a5bae1f0624b16be7940ac9b84c4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.718631 2648 tablet_service.cc:1467] Processing CreateTablet for tablet a3a5bae1f0624b16be7940ac9b84c4da (DEFAULT_TABLE table=default.Table_73c1ac5a47a045529546f8ff56205202 [id=5744abf74d4f405d9fc6a7e70804b513]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.725569 3075 ts_tablet_manager.cc:1428] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.031s user 0.002s sys 0.003s
I20260927 23:27:55.726027 3072 tablet_bootstrap.cc:492] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:55.726521 3072 ts_tablet_manager.cc:1397] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.183s user 0.008s sys 0.003s
I20260927 23:27:55.726747 2386 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 26d4c74dddca44bdbaac368e8d521fe7 (table default.Table_b906f759cf284e0aaf25a13094802a09 [id=dd99f8473f9a4f4f82b2127c1ea61fd2]). Expected version 1 got 0
I20260927 23:27:55.728670 3090 ts_tablet_manager.cc:1918] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:55.729058 3075 tablet_bootstrap.cc:492] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:55.729072 3090 log.cc:1196] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/502bdd29d2ab42579ae0239a7b6679c6
I20260927 23:27:55.732707 3090 ts_tablet_manager.cc:1939] T 502bdd29d2ab42579ae0239a7b6679c6 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:27:55.735546 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 410294c28c49404e9c547b27111712eb (DEFAULT_TABLE table=default.Table_9fcb69a219404bf4a763d08ffb0a744a [id=b215370f7b594ab28c3e2a34de2a6f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.736011 2948 tablet_service.cc:1467] Processing CreateTablet for tablet a3a5bae1f0624b16be7940ac9b84c4da (DEFAULT_TABLE table=default.Table_73c1ac5a47a045529546f8ff56205202 [id=5744abf74d4f405d9fc6a7e70804b513]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:55.738763 3075 tablet_bootstrap.cc:654] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
W20260927 23:27:55.739259 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_db0f806a353a4dbdb9d64635383bc826 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:55.740447 2390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_a2512a58a2fe47ff928e54a9b0a8f0f1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:55.742213 2390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a2512a58a2fe47ff928e54a9b0a8f0f1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:55.746315 3072 raft_consensus.cc:348] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.747133 3072 raft_consensus.cc:374] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.747414 3072 raft_consensus.cc:729] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
W20260927 23:27:55.747475 2761 leader_election.cc:341] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:55.749716 2386 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 26d4c74dddca44bdbaac368e8d521fe7 (table default.Table_b906f759cf284e0aaf25a13094802a09 [id=dd99f8473f9a4f4f82b2127c1ea61fd2]). Expected version 1 got 0
I20260927 23:27:55.749863 2387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_bc2a5a98bf2b443d999a2ecaa81e6309" } new_table_name: "default.Table_16715a5a56ca425399cded31536f5133" modify_external_catalogs: true
I20260927 23:27:55.759543 3079 raft_consensus.cc:504] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:55.761124 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 410294c28c49404e9c547b27111712eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.762930 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3004c09be14ebdbee12ce87c95e993" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:27:55.765184 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3a5bae1f0624b16be7940ac9b84c4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.769639 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3004c09be14ebdbee12ce87c95e993" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:55.770367 2828 raft_consensus.cc:3037] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:55.775776 3084 raft_consensus.cc:482] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:55.776353 3084 raft_consensus.cc:504] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.778538 3084 leader_election.cc:288] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:27:55.782027 2389 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2572268ca47346b0b78abc6a9ca7f908 consensus state has no leader
I20260927 23:27:55.784533 2828 raft_consensus.cc:2454] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:27:55.785252 3112 raft_consensus.cc:482] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:55.786069 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cfa2547e6a497a8db35651b03b9995" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:55.786803 2971 raft_consensus.cc:2454] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:27:55.788000 2761 leader_election.cc:302] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:27:55.788885 3084 raft_consensus.cc:2781] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:55.789194 3084 raft_consensus.cc:482] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:55.789491 3084 raft_consensus.cc:3037] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:55.790917 3077 raft_consensus.cc:504] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:55.788724 3112 raft_consensus.cc:504] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.796949 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 502bdd29d2ab42579ae0239a7b6679c6 (table default.Table_9e8530d40a5d49efae5ff9e54846bb86 [id=7895554af47d4b439fe421f68bb5a9a5]) successfully deleted
W20260927 23:27:55.798064 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet b5efb2e4760a48c98e1c6e9c1add0eca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b5efb2e4760a48c98e1c6e9c1add0eca already in progress: creating tablet
I20260927 23:27:55.799068 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3004c09be14ebdbee12ce87c95e993" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:55.800024 2674 raft_consensus.cc:2379] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:27:55.799774 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cfa2547e6a497a8db35651b03b9995" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:55.800518 2672 raft_consensus.cc:2454] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:27:55.801012 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cfa2547e6a497a8db35651b03b9995" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:55.801797 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3a5bae1f0624b16be7940ac9b84c4da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.802388 3070 tablet_bootstrap.cc:492] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:55.810890 3070 tablet_bootstrap.cc:654] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
W20260927 23:27:55.812862 2763 leader_election.cc:341] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 517d7bfc225a4372bfd79682a550a0d1
I20260927 23:27:55.813316 2763 leader_election.cc:302] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:55.814833 2609 leader_election.cc:302] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:55.814476 3112 leader_election.cc:288] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:55.815618 3112 raft_consensus.cc:2781] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:55.816567 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cfa2547e6a497a8db35651b03b9995" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:55.817299 2971 raft_consensus.cc:2454] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:55.824560 2608 leader_election.cc:302] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:27:55.828809 3084 raft_consensus.cc:504] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.830487 2827 raft_consensus.cc:2379] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5ce671f028a34ea4b8f0520c2689c9a1 in current term 1: Already voted for candidate abe182b2afc64d908f692ef45eae1d51 in this term.
I20260927 23:27:55.833070 3084 leader_election.cc:288] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:55.833236 3113 raft_consensus.cc:2726] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:55.834465 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cfa2547e6a497a8db35651b03b9995" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:27:55.835049 2971 raft_consensus.cc:3037] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:55.816087 3112 raft_consensus.cc:686] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:27:55.848037 3075 tablet_bootstrap.cc:492] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:55.848500 3075 ts_tablet_manager.cc:1397] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.120s user 0.016s sys 0.023s
I20260927 23:27:55.849465 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2572268ca47346b0b78abc6a9ca7f908. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.852264 3075 raft_consensus.cc:348] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.853010 3075 raft_consensus.cc:374] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.853251 3075 raft_consensus.cc:729] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.854334 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 410294c28c49404e9c547b27111712eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:55.853863 3075 consensus_queue.cc:260] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.857252 3075 ts_tablet_manager.cc:1428] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260927 23:27:55.858404 2971 raft_consensus.cc:2454] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:27:55.859953 3075 tablet_bootstrap.cc:492] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:55.865708 3075 tablet_bootstrap.cc:654] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.861765 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6cfa2547e6a497a8db35651b03b9995" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:27:55.872767 2674 raft_consensus.cc:3037] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:55.872629 2761 leader_election.cc:302] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:27:55.863879 3072 consensus_queue.cc:260] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.874127 3108 raft_consensus.cc:482] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:55.874562 3084 raft_consensus.cc:2781] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:55.874607 3108 raft_consensus.cc:504] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.875041 3084 raft_consensus.cc:686] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Becoming Leader. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Running, Role: LEADER
I20260927 23:27:55.876183 3084 consensus_queue.cc:237] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.881942 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5efb2e4760a48c98e1c6e9c1add0eca" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:27:55.883481 2906 leader_election.cc:341] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:55.885270 3108 leader_election.cc:288] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:55.893234 3112 consensus_queue.cc:237] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:55.898416 2972 raft_consensus.cc:2379] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5ce671f028a34ea4b8f0520c2689c9a1 in current term 1: Already voted for candidate de216a4a8e4b4e1b857d8647a3b05f6d in this term.
I20260927 23:27:55.907984 3072 ts_tablet_manager.cc:1428] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.181s user 0.008s sys 0.001s
W20260927 23:27:55.909956 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 95169.3us
I20260927 23:27:55.821455 3079 leader_election.cc:288] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:55.913508 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3004c09be14ebdbee12ce87c95e993" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:55.914369 2827 raft_consensus.cc:2379] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
W20260927 23:27:55.916420 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 97910.1us
I20260927 23:27:55.917845 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5efb2e4760a48c98e1c6e9c1add0eca" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:55.919229 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3004c09be14ebdbee12ce87c95e993" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
W20260927 23:27:55.920624 2907 leader_election.cc:341] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:55.921075 2907 leader_election.cc:302] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:55.922996 2907 leader_election.cc:302] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:55.924664 2388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet a3a5bae1f0624b16be7940ac9b84c4da
I20260927 23:27:55.929733 2652 tablet_service.cc:1514] Processing DeleteTablet for tablet a3a5bae1f0624b16be7940ac9b84c4da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:55.932173 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet a3a5bae1f0624b16be7940ac9b84c4da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:55.934082 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_bf2e1c7a89464e32b6c8230599344893"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:55.936602 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_bf2e1c7a89464e32b6c8230599344893 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:55.941958 3092 ts_tablet_manager.cc:1918] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:55.961066 3091 ts_tablet_manager.cc:1918] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:55.961529 3091 log.cc:1196] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/beaef767461647149eed1752cb18932c
I20260927 23:27:55.962190 3091 ts_tablet_manager.cc:1939] T beaef767461647149eed1752cb18932c P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:27:55.967983 2674 raft_consensus.cc:2454] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:27:55.974800 3114 raft_consensus.cc:2741] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260927 23:27:55.981175 3070 tablet_bootstrap.cc:492] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:55.981779 3070 ts_tablet_manager.cc:1397] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.180s user 0.008s sys 0.009s
I20260927 23:27:55.985275 3070 raft_consensus.cc:348] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.986446 3070 raft_consensus.cc:374] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:55.991616 3070 raft_consensus.cc:729] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:55.991737 3092 log.cc:1196] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/502bdd29d2ab42579ae0239a7b6679c6
I20260927 23:27:55.993359 3092 ts_tablet_manager.cc:1939] T 502bdd29d2ab42579ae0239a7b6679c6 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:27:56.000582 3075 tablet_bootstrap.cc:492] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:56.001084 3075 ts_tablet_manager.cc:1397] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.141s user 0.019s sys 0.044s
I20260927 23:27:56.003512 3075 raft_consensus.cc:348] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.004326 3075 raft_consensus.cc:374] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.004531 3075 raft_consensus.cc:729] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.008059 3075 consensus_queue.cc:260] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.010257 3075 ts_tablet_manager.cc:1428] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260927 23:27:56.011220 3075 tablet_bootstrap.cc:492] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:56.017917 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet beaef767461647149eed1752cb18932c (table default.Table_07b48e8af383404c80196f58f95ba9c2 [id=8ffc72b15ec6464ba1e10be014af68ed]) successfully deleted
I20260927 23:27:56.019367 3091 tablet_replica.cc:330] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:27:56.020727 3091 raft_consensus.cc:2229] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:56.024726 3091 raft_consensus.cc:2258] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:55.976542 2951 tablet_service.cc:1514] Processing DeleteTablet for tablet a3a5bae1f0624b16be7940ac9b84c4da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:56.023628 3070 consensus_queue.cc:260] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:55.976722 2949 tablet_service.cc:1514] Processing DeleteTablet for tablet b5efb2e4760a48c98e1c6e9c1add0eca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:56.035909 3075 tablet_bootstrap.cc:654] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.970435 3108 raft_consensus.cc:2726] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:56.039376 3090 tablet_replica.cc:330] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:27:56.039091 3091 ts_tablet_manager.cc:1905] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:56.040102 3070 ts_tablet_manager.cc:1428] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.058s user 0.004s sys 0.005s
I20260927 23:27:56.043864 3070 tablet_bootstrap.cc:492] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:56.045620 3108 raft_consensus.cc:2726] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260927 23:27:56.058874 3090 raft_consensus.cc:2229] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:56.060062 3075 tablet_bootstrap.cc:492] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:56.060549 3075 ts_tablet_manager.cc:1397] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.049s user 0.014s sys 0.000s
I20260927 23:27:56.061762 3090 raft_consensus.cc:2258] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:56.063258 3075 raft_consensus.cc:348] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.064211 3075 raft_consensus.cc:374] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.064484 3075 raft_consensus.cc:729] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.064615 3090 ts_tablet_manager.cc:1905] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:56.065248 3075 consensus_queue.cc:260] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.067428 3075 ts_tablet_manager.cc:1428] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260927 23:27:56.068512 3075 tablet_bootstrap.cc:492] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:56.074818 3075 tablet_bootstrap.cc:654] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:55.959228 3077 leader_election.cc:288] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:56.075830 3072 tablet_bootstrap.cc:492] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:56.095285 3070 tablet_bootstrap.cc:654] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:56.102676 3075 tablet_bootstrap.cc:492] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:56.103158 3075 ts_tablet_manager.cc:1397] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.035s user 0.005s sys 0.007s
I20260927 23:27:56.105439 3075 raft_consensus.cc:348] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.106045 3075 raft_consensus.cc:374] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.106213 3075 raft_consensus.cc:729] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.116632 3075 consensus_queue.cc:260] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.118522 3091 ts_tablet_manager.cc:1918] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:56.119190 3075 ts_tablet_manager.cc:1428] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.016s user 0.005s sys 0.001s
I20260927 23:27:56.120134 3075 tablet_bootstrap.cc:492] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:56.122113 3091 log.cc:1196] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/502bdd29d2ab42579ae0239a7b6679c6
I20260927 23:27:56.124337 3112 raft_consensus.cc:482] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:56.121770 3070 tablet_bootstrap.cc:492] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:56.124992 3070 ts_tablet_manager.cc:1397] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.081s user 0.006s sys 0.009s
I20260927 23:27:56.124790 3112 raft_consensus.cc:504] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.126914 3112 leader_election.cc:288] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:56.127401 3091 ts_tablet_manager.cc:1939] T 502bdd29d2ab42579ae0239a7b6679c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:27:56.133489 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d4c74dddca44bdbaac368e8d521fe7" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:56.128516 3070 raft_consensus.cc:348] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.134339 3070 raft_consensus.cc:374] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.134661 3070 raft_consensus.cc:729] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.135093 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d4c74dddca44bdbaac368e8d521fe7" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:56.135437 3075 tablet_bootstrap.cc:654] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:56.135433 3070 consensus_queue.cc:260] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.135903 2827 raft_consensus.cc:2454] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:56.139560 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 502bdd29d2ab42579ae0239a7b6679c6 (table default.Table_9e8530d40a5d49efae5ff9e54846bb86 [id=7895554af47d4b439fe421f68bb5a9a5]) successfully deleted
I20260927 23:27:56.140650 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 502bdd29d2ab42579ae0239a7b6679c6 (table default.Table_9e8530d40a5d49efae5ff9e54846bb86 [id=7895554af47d4b439fe421f68bb5a9a5]) successfully deleted
W20260927 23:27:56.137835 2608 leader_election.cc:341] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.143059 2609 leader_election.cc:302] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:56.144212 3112 raft_consensus.cc:2781] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:56.144593 3112 raft_consensus.cc:482] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:56.144879 3112 raft_consensus.cc:3037] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:56.152419 2388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_b906f759cf284e0aaf25a13094802a09" } modify_external_catalogs: true
I20260927 23:27:56.153147 2388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_b906f759cf284e0aaf25a13094802a09" } modify_external_catalogs: true
I20260927 23:27:56.154415 3112 raft_consensus.cc:504] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.158701 2393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_0b552fe90a6342eeb8c086f4c7436f07" } new_table_name: "default.Table_35a03bcad08c4020b92156f715f3cdef" modify_external_catalogs: true
I20260927 23:27:56.159901 3075 tablet_bootstrap.cc:492] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:56.160352 3075 ts_tablet_manager.cc:1397] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.040s user 0.007s sys 0.009s
I20260927 23:27:56.163399 3075 raft_consensus.cc:348] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.164165 3075 raft_consensus.cc:374] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.164448 3075 raft_consensus.cc:729] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.167586 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d4c74dddca44bdbaac368e8d521fe7" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:56.168201 2827 raft_consensus.cc:3037] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:56.175035 2827 raft_consensus.cc:2454] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:27:56.176604 3075 consensus_queue.cc:260] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:27:56.177138 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet b5efb2e4760a48c98e1c6e9c1add0eca because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b5efb2e4760a48c98e1c6e9c1add0eca already in progress: creating tablet
I20260927 23:27:56.179826 3075 ts_tablet_manager.cc:1428] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260927 23:27:56.180179 2609 leader_election.cc:302] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:27:56.181067 3114 raft_consensus.cc:2781] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:56.181545 3114 raft_consensus.cc:686] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:27:56.182385 3114 consensus_queue.cc:237] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:27:56.189985 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet a3a5bae1f0624b16be7940ac9b84c4da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3a5bae1f0624b16be7940ac9b84c4da already in progress: creating tablet
I20260927 23:27:56.195307 3070 ts_tablet_manager.cc:1428] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.070s user 0.006s sys 0.000s
I20260927 23:27:56.199283 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26d4c74dddca44bdbaac368e8d521fe7" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:27:56.200412 3070 tablet_bootstrap.cc:492] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:56.200325 3084 raft_consensus.cc:482] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:56.200796 3084 raft_consensus.cc:504] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.201323 3112 leader_election.cc:288] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:56.202821 3084 leader_election.cc:288] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:56.204538 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4372ae7a9d6426a97419d89253f99d1" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:27:56.205911 2763 leader_election.cc:341] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:27:56.206945 2608 leader_election.cc:341] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.210937 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4372ae7a9d6426a97419d89253f99d1" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:56.211017 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_506a46f561d84a06a787e2d54539696d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:56.212901 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_506a46f561d84a06a787e2d54539696d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:27:56.212898 2761 leader_election.cc:341] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.213431 2761 leader_election.cc:302] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:56.214247 3084 raft_consensus.cc:2726] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:56.218963 2390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_3597d6af5e24454ab687a0c4be731a86"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:56.221027 2390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3597d6af5e24454ab687a0c4be731a86 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:56.223037 3070 tablet_bootstrap.cc:654] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:56.228892 3092 tablet_replica.cc:330] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:27:56.229934 3092 raft_consensus.cc:2229] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:56.230575 3092 raft_consensus.cc:2258] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:56.233387 3092 ts_tablet_manager.cc:1905] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:56.253062 3070 tablet_bootstrap.cc:492] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
W20260927 23:27:56.252970 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 330351us
I20260927 23:27:56.253616 3070 ts_tablet_manager.cc:1397] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.053s user 0.010s sys 0.003s
I20260927 23:27:56.256891 3072 tablet_bootstrap.cc:700] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent opening tablet: real 0.165s user 0.004s sys 0.000s
I20260927 23:27:56.257809 3072 tablet_bootstrap.cc:654] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:56.273676 3070 raft_consensus.cc:348] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.274478 3070 raft_consensus.cc:374] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.274735 3070 raft_consensus.cc:729] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.275228 3084 raft_consensus.cc:482] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:56.275714 3084 raft_consensus.cc:504] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.277799 3084 leader_election.cc:288] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:56.279597 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3a5bae1f0624b16be7940ac9b84c4da" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:27:56.282235 2763 leader_election.cc:341] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.284072 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3a5bae1f0624b16be7940ac9b84c4da" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:56.284102 3070 consensus_queue.cc:260] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:27:56.285995 2761 leader_election.cc:341] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.289335 2761 leader_election.cc:302] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:56.290174 3070 ts_tablet_manager.cc:1428] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.036s user 0.007s sys 0.001s
I20260927 23:27:56.290211 3084 raft_consensus.cc:2726] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:56.296069 3070 tablet_bootstrap.cc:492] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:56.306288 3084 raft_consensus.cc:482] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:56.306957 3084 raft_consensus.cc:504] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.309124 3084 leader_election.cc:288] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:56.311077 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2572268ca47346b0b78abc6a9ca7f908" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:56.313308 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2572268ca47346b0b78abc6a9ca7f908" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:27:56.315387 2761 leader_election.cc:341] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:27:56.317421 2763 leader_election.cc:341] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.317806 2763 leader_election.cc:302] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:56.318928 3084 raft_consensus.cc:2726] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:56.325875 3070 tablet_bootstrap.cc:654] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:56.351117 2649 tablet_service.cc:1467] Processing CreateTablet for tablet a731a36b1da24f559c77bd2b6b484c72 (DEFAULT_TABLE table=default.Table_f4808e8ee6d64bbb8e824ed11976279d [id=94bf26c26f974c258ee084dbc165be72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:56.352444 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a731a36b1da24f559c77bd2b6b484c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:56.353842 2804 tablet_service.cc:1467] Processing CreateTablet for tablet a731a36b1da24f559c77bd2b6b484c72 (DEFAULT_TABLE table=default.Table_f4808e8ee6d64bbb8e824ed11976279d [id=94bf26c26f974c258ee084dbc165be72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:56.353037 2385 catalog_manager.cc:5495] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:27:56.355046 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a731a36b1da24f559c77bd2b6b484c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:56.354987 2385 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet af3004c09be14ebdbee12ce87c95e993 (table default.Table_bc2a5a98bf2b443d999a2ecaa81e6309 [id=c1a377dfa52a4e62b53dc8ffbe3ae6dc]). Expected version 2 got 0
I20260927 23:27:56.358150 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d (DEFAULT_TABLE table=default.Table_5f321d8241ab4a69a82e325e59605ab2 [id=c227fcbf71824cb49a4c3061117fed02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:56.359349 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:56.360034 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d (DEFAULT_TABLE table=default.Table_5f321d8241ab4a69a82e325e59605ab2 [id=c227fcbf71824cb49a4c3061117fed02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:56.361189 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:56.365775 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet a4372ae7a9d6426a97419d89253f99d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:56.373740 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d (DEFAULT_TABLE table=default.Table_5f321d8241ab4a69a82e325e59605ab2 [id=c227fcbf71824cb49a4c3061117fed02]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:27:56.375466 2387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet af3004c09be14ebdbee12ce87c95e993 consensus state has no leader
I20260927 23:27:56.356371 2392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet a4372ae7a9d6426a97419d89253f99d1
I20260927 23:27:56.378059 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet a4372ae7a9d6426a97419d89253f99d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:27:56.379788 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet a4372ae7a9d6426a97419d89253f99d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4372ae7a9d6426a97419d89253f99d1 already in progress: creating tablet
I20260927 23:27:56.382392 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet a4372ae7a9d6426a97419d89253f99d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:56.380251 2951 tablet_service.cc:1514] Processing DeleteTablet for tablet a4372ae7a9d6426a97419d89253f99d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:56.388993 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:56.387396 2950 tablet_service.cc:1467] Processing CreateTablet for tablet a731a36b1da24f559c77bd2b6b484c72 (DEFAULT_TABLE table=default.Table_f4808e8ee6d64bbb8e824ed11976279d [id=94bf26c26f974c258ee084dbc165be72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:56.394773 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a731a36b1da24f559c77bd2b6b484c72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:56.417155 3072 tablet_bootstrap.cc:492] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:56.417655 3072 ts_tablet_manager.cc:1397] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.342s user 0.017s sys 0.000s
I20260927 23:27:56.421525 3075 tablet_bootstrap.cc:492] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:56.426833 3075 tablet_bootstrap.cc:654] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:56.429141 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_5028982a958d4137a4c4d62e91241d32"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:56.430992 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5028982a958d4137a4c4d62e91241d32 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:56.438881 2392 catalog_manager.cc:5495] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe182b2afc64d908f692ef45eae1d51 (127.2.57.2). New cstate: current_term: 1 leader_uuid: "abe182b2afc64d908f692ef45eae1d51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:27:56.444139 2392 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet d6cfa2547e6a497a8db35651b03b9995 (table default.Table_0b552fe90a6342eeb8c086f4c7436f07 [id=18c041f9985b4c9f99cbc8cc4f74db94]). Expected version 2 got 0
I20260927 23:27:56.445226 2392 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 2572268ca47346b0b78abc6a9ca7f908 (table default.Table_2d889ed8e18e47a79f0541db93242386 [id=103d7c5e210c48fdafb8c199eb4cf676]). Expected version 1 got 0
W20260927 23:27:56.450927 2393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet d6cfa2547e6a497a8db35651b03b9995 consensus state has no leader
I20260927 23:27:56.459522 3090 ts_tablet_manager.cc:1918] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:56.460037 3090 log.cc:1196] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/b5efb2e4760a48c98e1c6e9c1add0eca
I20260927 23:27:56.462384 3070 tablet_bootstrap.cc:492] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:56.462837 3070 ts_tablet_manager.cc:1397] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.167s user 0.015s sys 0.046s
I20260927 23:27:56.469424 3114 raft_consensus.cc:482] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:56.469960 3114 raft_consensus.cc:504] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.473621 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "517d7bfc225a4372bfd79682a550a0d1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:56.474314 2827 raft_consensus.cc:2454] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:56.476212 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "517d7bfc225a4372bfd79682a550a0d1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:56.477825 2609 leader_election.cc:302] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:27:56.478852 3112 raft_consensus.cc:2781] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260927 23:27:56.479144 2608 leader_election.cc:341] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.479179 3112 raft_consensus.cc:482] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:56.479931 3112 raft_consensus.cc:3037] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:56.482777 3072 raft_consensus.cc:348] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.483518 3072 raft_consensus.cc:374] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.483765 3072 raft_consensus.cc:729] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.472067 3114 leader_election.cc:288] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:56.488351 3112 raft_consensus.cc:504] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.477748 3070 raft_consensus.cc:348] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.489697 3070 raft_consensus.cc:374] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.489959 3070 raft_consensus.cc:729] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.492734 3075 tablet_bootstrap.cc:492] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:56.490545 3070 consensus_queue.cc:260] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.493180 3075 ts_tablet_manager.cc:1397] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.072s user 0.014s sys 0.005s
I20260927 23:27:56.494652 3070 ts_tablet_manager.cc:1428] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.032s user 0.006s sys 0.001s
I20260927 23:27:56.495672 3070 tablet_bootstrap.cc:492] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:56.499428 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "517d7bfc225a4372bfd79682a550a0d1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:27:56.500944 3090 ts_tablet_manager.cc:1939] T b5efb2e4760a48c98e1c6e9c1add0eca P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
W20260927 23:27:56.502556 2608 leader_election.cc:341] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.504168 3112 leader_election.cc:288] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:56.497093 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "517d7bfc225a4372bfd79682a550a0d1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:56.504696 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet b5efb2e4760a48c98e1c6e9c1add0eca (table default.Table_e3fad1f8c2494735b4723d0ecc1b0b39 [id=601abf06edf443879124d16259163a7e]) successfully deleted
I20260927 23:27:56.504869 2827 raft_consensus.cc:3037] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:56.511797 2827 raft_consensus.cc:2454] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:27:56.512998 2609 leader_election.cc:302] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:56.515420 2388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 26d4c74dddca44bdbaac368e8d521fe7
I20260927 23:27:56.516494 3075 raft_consensus.cc:348] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.517185 3075 raft_consensus.cc:374] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.517439 3075 raft_consensus.cc:729] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.518391 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d4c74dddca44bdbaac368e8d521fe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:56.518988 3112 raft_consensus.cc:2781] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:56.519196 3091 tablet_replica.cc:330] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:27:56.519732 3112 raft_consensus.cc:686] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:27:56.519989 3091 raft_consensus.cc:2229] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:27:56.520870 3091 pending_rounds.cc:62] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1: Trying to abort 1 pending ops.
I20260927 23:27:56.520592 3112 consensus_queue.cc:237] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.521116 3091 pending_rounds.cc:68] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099665659727872 op_type: NO_OP noop_request { }
I20260927 23:27:56.521268 2951 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d4c74dddca44bdbaac368e8d521fe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:56.521579 3091 raft_consensus.cc:2866] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:27:56.521978 3091 raft_consensus.cc:2258] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:56.522037 2805 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d4c74dddca44bdbaac368e8d521fe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:56.525399 3091 ts_tablet_manager.cc:1905] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:56.518045 3075 consensus_queue.cc:260] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.541275 3075 ts_tablet_manager.cc:1428] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.027s user 0.005s sys 0.000s
I20260927 23:27:56.546021 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_faba7f79481947c1a44139eddeb749a6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:56.547924 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_faba7f79481947c1a44139eddeb749a6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:56.550359 3075 tablet_bootstrap.cc:492] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:56.567548 3075 tablet_bootstrap.cc:654] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:56.556119 3072 consensus_queue.cc:260] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:27:56.612812 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet a3a5bae1f0624b16be7940ac9b84c4da because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a3a5bae1f0624b16be7940ac9b84c4da already in progress: creating tablet
I20260927 23:27:56.606010 3091 ts_tablet_manager.cc:1918] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
W20260927 23:27:56.613911 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet a4372ae7a9d6426a97419d89253f99d1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4372ae7a9d6426a97419d89253f99d1 already in progress: creating tablet
I20260927 23:27:56.614295 3091 log.cc:1196] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/26d4c74dddca44bdbaac368e8d521fe7
W20260927 23:27:56.614850 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 26d4c74dddca44bdbaac368e8d521fe7 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26d4c74dddca44bdbaac368e8d521fe7 already in progress: creating tablet
I20260927 23:27:56.615157 3091 ts_tablet_manager.cc:1939] T 26d4c74dddca44bdbaac368e8d521fe7 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:27:56.617518 3072 ts_tablet_manager.cc:1428] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.137s user 0.009s sys 0.000s
I20260927 23:27:56.625208 2390 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_8166baf5795042f5b79dd39a7d545027" } new_table_name: "default.Table_c42131f0bbf640bbbf4b8dc955fcd001" modify_external_catalogs: true
I20260927 23:27:56.629352 2391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_16715a5a56ca425399cded31536f5133" } modify_external_catalogs: true
I20260927 23:27:56.630877 2391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_16715a5a56ca425399cded31536f5133" } modify_external_catalogs: true
I20260927 23:27:56.633935 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 26d4c74dddca44bdbaac368e8d521fe7 (table default.Table_b906f759cf284e0aaf25a13094802a09 [id=dd99f8473f9a4f4f82b2127c1ea61fd2]) successfully deleted
I20260927 23:27:56.638710 3070 tablet_bootstrap.cc:654] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:56.640758 2972 raft_consensus.cc:1261] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Refusing update from remote peer 5ce671f028a34ea4b8f0520c2689c9a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:27:56.641232 2827 raft_consensus.cc:1261] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Refusing update from remote peer 5ce671f028a34ea4b8f0520c2689c9a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:27:56.642946 3114 consensus_queue.cc:1035] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:27:56.644781 3114 consensus_queue.cc:1035] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:27:56.650008 3072 tablet_bootstrap.cc:492] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:56.677268 3075 tablet_bootstrap.cc:492] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:56.677735 3075 ts_tablet_manager.cc:1397] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.128s user 0.021s sys 0.026s
I20260927 23:27:56.674469 3141 tablet.cc:1722] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260927 23:27:56.680742 3136 mvcc.cc:205] Tried to move back new op lower bound from 7334099667496505344 to 7334099664589053952. Current Snapshot: MvccSnapshot[applied={T|T < 7334099667496505344}]
I20260927 23:27:56.683050 3139 mvcc.cc:205] Tried to move back new op lower bound from 7334099667496505344 to 7334099664589053952. Current Snapshot: MvccSnapshot[applied={T|T < 7334099667496505344}]
I20260927 23:27:56.684500 3112 raft_consensus.cc:482] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:56.682539 3075 raft_consensus.cc:348] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.685217 2393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_5028982a958d4137a4c4d62e91241d32" } new_table_name: "default.Table_877402b3ef234b9c99bbdab0842b3885" modify_external_catalogs: true
I20260927 23:27:56.685724 3075 raft_consensus.cc:374] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.686645 3112 raft_consensus.cc:504] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.686082 3075 raft_consensus.cc:729] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.690448 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4372ae7a9d6426a97419d89253f99d1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:56.691179 2826 raft_consensus.cc:2454] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:56.692508 2609 leader_election.cc:302] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:27:56.693363 3114 raft_consensus.cc:2781] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:56.693694 3114 raft_consensus.cc:482] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:56.689764 3075 consensus_queue.cc:260] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.693602 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_0a453cae4de74eb2992e691b44d18700"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:56.693991 3114 raft_consensus.cc:3037] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:27:56.695399 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0a453cae4de74eb2992e691b44d18700 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:56.695045 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_3963531cde4641e785245322c0a23fab"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:56.698395 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3963531cde4641e785245322c0a23fab in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:56.698861 3075 ts_tablet_manager.cc:1428] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.021s user 0.002s sys 0.004s
I20260927 23:27:56.702499 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4372ae7a9d6426a97419d89253f99d1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:27:56.704126 2608 leader_election.cc:341] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.706218 3112 leader_election.cc:288] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:56.711870 3092 ts_tablet_manager.cc:1918] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:56.712795 3092 log.cc:1196] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/b5efb2e4760a48c98e1c6e9c1add0eca
I20260927 23:27:56.713934 3092 ts_tablet_manager.cc:1939] T b5efb2e4760a48c98e1c6e9c1add0eca P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:27:56.715268 3072 tablet_bootstrap.cc:654] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:56.725453 3142 tablet.cc:1722] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260927 23:27:56.727221 2384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_35a03bcad08c4020b92156f715f3cdef" } modify_external_catalogs: true
I20260927 23:27:56.727855 2384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_35a03bcad08c4020b92156f715f3cdef" } modify_external_catalogs: true
I20260927 23:27:56.711784 3114 raft_consensus.cc:504] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.734274 3070 tablet_bootstrap.cc:492] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:56.734807 3070 ts_tablet_manager.cc:1397] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.239s user 0.007s sys 0.005s
I20260927 23:27:56.735833 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4372ae7a9d6426a97419d89253f99d1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
W20260927 23:27:56.737835 2608 leader_election.cc:341] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.738672 3114 leader_election.cc:288] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:56.739275 3070 raft_consensus.cc:348] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.739976 3070 raft_consensus.cc:374] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.740247 3070 raft_consensus.cc:729] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.741101 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4372ae7a9d6426a97419d89253f99d1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:56.740927 3070 consensus_queue.cc:260] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.741691 2828 raft_consensus.cc:3037] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:56.746358 3092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 13) took 1243 ms. Trace:
I20260927 23:27:56.746759 3092 rpcz_store.cc:276] 0927 23:27:55.502701 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:55.502865 (+ 164us) service_pool.cc:227] Handling call
0927 23:27:56.228688 (+725823us) ts_tablet_manager.cc:1145] Deleting tablet b5efb2e4760a48c98e1c6e9c1add0eca
0927 23:27:56.228748 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:27:56.536969 (+308221us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:56.711460 (+174491us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:56.746276 (+ 34816us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":725135}
I20260927 23:27:56.748509 3092 tablet_replica.cc:330] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:27:56.749300 3092 raft_consensus.cc:2229] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:56.749787 3092 raft_consensus.cc:2258] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260927 23:27:56.755419 2379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 26d4c74dddca44bdbaac368e8d521fe7 (table default.Table_b906f759cf284e0aaf25a13094802a09 [id=dd99f8473f9a4f4f82b2127c1ea61fd2], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 26d4c74dddca44bdbaac368e8d521fe7 consensus state has no leader
I20260927 23:27:56.758250 2828 raft_consensus.cc:2454] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:27:56.762202 2609 leader_election.cc:302] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:56.764559 3114 raft_consensus.cc:2781] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:56.765357 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet b5efb2e4760a48c98e1c6e9c1add0eca (table default.Table_e3fad1f8c2494735b4723d0ecc1b0b39 [id=601abf06edf443879124d16259163a7e]) successfully deleted
I20260927 23:27:56.767445 3092 ts_tablet_manager.cc:1905] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:56.768749 3070 ts_tablet_manager.cc:1428] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.034s user 0.009s sys 0.000s
I20260927 23:27:56.769738 3070 tablet_bootstrap.cc:492] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:56.778775 3072 tablet_bootstrap.cc:492] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:56.779306 3072 ts_tablet_manager.cc:1397] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.129s user 0.009s sys 0.003s
I20260927 23:27:56.776867 2385 catalog_manager.cc:5495] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
W20260927 23:27:56.781407 2390 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 517d7bfc225a4372bfd79682a550a0d1 consensus state has no leader
I20260927 23:27:56.782619 2391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet af3004c09be14ebdbee12ce87c95e993
I20260927 23:27:56.782600 3072 raft_consensus.cc:348] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.783421 3072 raft_consensus.cc:374] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.783667 3072 raft_consensus.cc:729] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.786180 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet af3004c09be14ebdbee12ce87c95e993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:56.786979 3090 tablet_replica.cc:330] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:27:56.772393 3114 raft_consensus.cc:686] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:27:56.797515 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet af3004c09be14ebdbee12ce87c95e993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:56.798283 3091 tablet_replica.cc:330] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:27:56.799152 3091 raft_consensus.cc:2229] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:27:56.800127 3091 raft_consensus.cc:2258] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:56.797348 3114 consensus_queue.cc:237] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.802405 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet af3004c09be14ebdbee12ce87c95e993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:56.803097 3084 raft_consensus.cc:482] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:56.803637 3084 raft_consensus.cc:504] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.806058 3084 leader_election.cc:288] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:56.808143 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60a2bd7ab1bc4fc29cc4e70d6e6de70d" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:56.808255 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60a2bd7ab1bc4fc29cc4e70d6e6de70d" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:27:56.809679 2761 leader_election.cc:341] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:27:56.810221 2763 leader_election.cc:341] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.810626 2763 leader_election.cc:302] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:56.811591 3084 raft_consensus.cc:2726] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:56.811457 3070 tablet_bootstrap.cc:654] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:56.812319 3072 consensus_queue.cc:260] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:56.820729 2674 raft_consensus.cc:1261] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Refusing update from remote peer abe182b2afc64d908f692ef45eae1d51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:27:56.820818 2971 raft_consensus.cc:1261] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Refusing update from remote peer abe182b2afc64d908f692ef45eae1d51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:27:56.822973 3084 consensus_queue.cc:1035] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:27:56.829607 3084 consensus_queue.cc:1035] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260927 23:27:56.840210 2393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 has no consensus state
I20260927 23:27:56.854152 3141 tablet.cc:1722] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260927 23:27:56.857539 3091 ts_tablet_manager.cc:1905] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:56.860862 3072 ts_tablet_manager.cc:1428] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.081s user 0.009s sys 0.000s
I20260927 23:27:56.865165 3072 tablet_bootstrap.cc:492] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:56.866182 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_1f563f833bd94200a45952661e37e1fe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:56.868072 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1f563f833bd94200a45952661e37e1fe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:56.893215 3090 raft_consensus.cc:2229] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:56.894654 3090 pending_rounds.cc:62] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: Trying to abort 2 pending ops.
I20260927 23:27:56.895893 3090 pending_rounds.cc:68] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7334099667496505344 op_type: ALTER_SCHEMA_OP alter_schema_request { tablet_id: "af3004c09be14ebdbee12ce87c95e993" schema { columns { id: 10 name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } schema_version: 3 new_table_name: "default.Table_16715a5a56ca425399cded31536f5133" dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" new_extra_config { } }
I20260927 23:27:56.898356 3090 pending_rounds.cc:68] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099664589053952 op_type: NO_OP noop_request { }
I20260927 23:27:56.898801 3090 raft_consensus.cc:2866] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:27:56.899212 3090 raft_consensus.cc:2258] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:56.905241 3142 tablet.cc:1722] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260927 23:27:56.907301 3147 raft_consensus.cc:482] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:56.908284 3147 raft_consensus.cc:504] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.911722 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410294c28c49404e9c547b27111712eb" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:27:56.913398 2761 leader_election.cc:341] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.916728 3092 ts_tablet_manager.cc:1918] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:56.917133 3092 log.cc:1196] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/a3a5bae1f0624b16be7940ac9b84c4da
I20260927 23:27:56.916944 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410294c28c49404e9c547b27111712eb" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:56.917804 3092 ts_tablet_manager.cc:1939] T a3a5bae1f0624b16be7940ac9b84c4da P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:27:56.918498 3147 leader_election.cc:288] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:27:56.918499 2763 leader_election.cc:341] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:56.919384 2763 leader_election.cc:302] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:56.931417 3147 raft_consensus.cc:2726] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:27:56.938064 3145 consensus_queue.cc:1159] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260927 23:27:56.941246 3070 tablet_bootstrap.cc:492] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:56.941574 3090 ts_tablet_manager.cc:1905] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:56.941954 3070 ts_tablet_manager.cc:1397] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.172s user 0.011s sys 0.006s
I20260927 23:27:56.944324 3070 raft_consensus.cc:348] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.945119 3070 raft_consensus.cc:374] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:56.945416 3070 raft_consensus.cc:729] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:56.946123 3070 consensus_queue.cc:260] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:56.953682 3070 ts_tablet_manager.cc:1428] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260927 23:27:56.959496 3070 tablet_bootstrap.cc:492] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:56.966912 3070 tablet_bootstrap.cc:654] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:56.961193 3091 ts_tablet_manager.cc:1918] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:27:56.973724 3091 log.cc:1196] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/af3004c09be14ebdbee12ce87c95e993
I20260927 23:27:56.974714 3091 ts_tablet_manager.cc:1939] T af3004c09be14ebdbee12ce87c95e993 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:27:56.977691 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet af3004c09be14ebdbee12ce87c95e993 (table default.Table_16715a5a56ca425399cded31536f5133 [id=c1a377dfa52a4e62b53dc8ffbe3ae6dc]) successfully deleted
I20260927 23:27:56.985226 3114 raft_consensus.cc:482] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:56.986075 2384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet d6cfa2547e6a497a8db35651b03b9995
I20260927 23:27:56.985688 3114 raft_consensus.cc:504] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:27:56.989267 2609 leader_election.cc:341] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Not found: Tablet not found: b5efb2e4760a48c98e1c6e9c1add0eca
I20260927 23:27:56.990598 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cfa2547e6a497a8db35651b03b9995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:56.991120 3114 leader_election.cc:288] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:27:56.991381 3091 tablet_replica.cc:330] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:27:56.992139 3091 raft_consensus.cc:2229] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:56.992897 3091 raft_consensus.cc:2258] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 23:27:56.993965 2608 leader_election.cc:341] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: b5efb2e4760a48c98e1c6e9c1add0eca
I20260927 23:27:56.994369 2608 leader_election.cc:302] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:56.995348 3114 raft_consensus.cc:2726] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:56.998937 3070 tablet_bootstrap.cc:492] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:56.999359 3070 ts_tablet_manager.cc:1397] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.040s user 0.007s sys 0.004s
I20260927 23:27:57.001818 3070 raft_consensus.cc:348] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.002434 3070 raft_consensus.cc:374] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:57.002677 3070 raft_consensus.cc:729] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:57.008093 3072 tablet_bootstrap.cc:654] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:57.007117 3091 ts_tablet_manager.cc:1905] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:57.012624 3070 consensus_queue.cc:260] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.014647 3092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 17) took 1083 ms. Trace:
I20260927 23:27:57.015039 3092 rpcz_store.cc:276] 0927 23:27:55.931765 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:55.931960 (+ 195us) service_pool.cc:227] Handling call
0927 23:27:56.748333 (+816373us) ts_tablet_manager.cc:1145] Deleting tablet a3a5bae1f0624b16be7940ac9b84c4da
0927 23:27:56.748381 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:27:56.851480 (+103099us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:56.916473 (+ 64993us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:57.014562 (+ 98089us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":815592}
I20260927 23:27:57.018585 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet a3a5bae1f0624b16be7940ac9b84c4da (table default.Table_73c1ac5a47a045529546f8ff56205202 [id=5744abf74d4f405d9fc6a7e70804b513]) successfully deleted
I20260927 23:27:57.020444 3070 ts_tablet_manager.cc:1428] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.021s user 0.006s sys 0.002s
I20260927 23:27:57.023434 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cfa2547e6a497a8db35651b03b9995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:57.024103 3070 tablet_bootstrap.cc:492] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:57.018790 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cfa2547e6a497a8db35651b03b9995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:57.019028 2805 tablet_service.cc:1514] Processing DeleteTablet for tablet b5efb2e4760a48c98e1c6e9c1add0eca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:57.016265 3092 tablet_replica.cc:330] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:27:57.030272 3070 tablet_bootstrap.cc:654] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:57.030727 3092 raft_consensus.cc:2229] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:57.031333 3092 raft_consensus.cc:2258] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:57.033391 3092 ts_tablet_manager.cc:1905] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:57.034787 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet b5efb2e4760a48c98e1c6e9c1add0eca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:57.035152 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet af3004c09be14ebdbee12ce87c95e993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:57.036561 2652 tablet_service.cc:1514] Processing DeleteTablet for tablet a4372ae7a9d6426a97419d89253f99d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:57.036036 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d4c74dddca44bdbaac368e8d521fe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:57.056559 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet 26d4c74dddca44bdbaac368e8d521fe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:57.074277 3092 ts_tablet_manager.cc:1918] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:57.074740 3092 log.cc:1196] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/a4372ae7a9d6426a97419d89253f99d1
I20260927 23:27:57.075433 3092 ts_tablet_manager.cc:1939] T a4372ae7a9d6426a97419d89253f99d1 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:27:57.079312 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet a4372ae7a9d6426a97419d89253f99d1 (table default.Table_1b6c64cbb7304c48a43dfa5cef404376 [id=5f884e6c6e6445bbbbdeacc26cd347a5]) successfully deleted
I20260927 23:27:57.080700 3154 tablet.cc:1722] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
W20260927 23:27:57.086319 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet a4372ae7a9d6426a97419d89253f99d1 because the tablet was not found. No further retry: Not found: Tablet not found: a4372ae7a9d6426a97419d89253f99d1
I20260927 23:27:57.086247 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_42b58587cafe46998c02643a817f30be"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:57.087452 3092 tablet_replica.cc:330] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:27:57.088418 3092 raft_consensus.cc:2229] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:57.088963 3092 raft_consensus.cc:2258] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:57.092379 2391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_d8ee874db95045efa6c1321ce207e4ea" } new_table_name: "default.Table_44775a3339af4c55ae5f701b0e17b7a1" modify_external_catalogs: true
I20260927 23:27:57.095785 3092 ts_tablet_manager.cc:1905] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:27:57.099464 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_42b58587cafe46998c02643a817f30be in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:57.080281 3090 ts_tablet_manager.cc:1918] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:27:57.102072 3090 log.cc:1196] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/af3004c09be14ebdbee12ce87c95e993
I20260927 23:27:57.102885 3090 ts_tablet_manager.cc:1939] T af3004c09be14ebdbee12ce87c95e993 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:27:57.110798 2384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_5f321d8241ab4a69a82e325e59605ab2" } modify_external_catalogs: true
I20260927 23:27:57.111555 2384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_5f321d8241ab4a69a82e325e59605ab2" } modify_external_catalogs: true
I20260927 23:27:57.119997 3137 raft_consensus.cc:482] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:57.120595 3137 raft_consensus.cc:504] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:57.123575 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet af3004c09be14ebdbee12ce87c95e993 (table default.Table_16715a5a56ca425399cded31536f5133 [id=c1a377dfa52a4e62b53dc8ffbe3ae6dc]) successfully deleted
I20260927 23:27:57.124595 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "517d7bfc225a4372bfd79682a550a0d1" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:57.126147 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "517d7bfc225a4372bfd79682a550a0d1" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:57.127054 2828 raft_consensus.cc:2379] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:27:57.128367 3137 leader_election.cc:288] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:57.132985 3090 tablet_replica.cc:330] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:27:57.133396 2907 leader_election.cc:302] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:57.133802 3090 raft_consensus.cc:2229] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:57.134351 3090 raft_consensus.cc:2258] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:57.134634 3137 raft_consensus.cc:3037] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:57.139439 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_e7672fd6100045e3945ce7ddd03edbf1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:57.141376 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e7672fd6100045e3945ce7ddd03edbf1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:57.143240 3137 raft_consensus.cc:2726] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:57.152338 2828 raft_consensus.cc:1261] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Refusing update from remote peer 5ce671f028a34ea4b8f0520c2689c9a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:27:57.154743 3112 consensus_queue.cc:1035] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:27:57.160576 3090 ts_tablet_manager.cc:1905] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:57.161515 2392 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_506a46f561d84a06a787e2d54539696d" } new_table_name: "default.Table_35e97e4b9b9c4dc49db9e1970f335204" modify_external_catalogs: true
I20260927 23:27:57.169878 2971 raft_consensus.cc:1261] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Refusing update from remote peer 5ce671f028a34ea4b8f0520c2689c9a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:27:57.170435 3091 ts_tablet_manager.cc:1918] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:27:57.170917 3091 log.cc:1196] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/d6cfa2547e6a497a8db35651b03b9995
I20260927 23:27:57.171715 3091 ts_tablet_manager.cc:1939] T d6cfa2547e6a497a8db35651b03b9995 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:27:57.177384 3141 tablet.cc:1722] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:27:57.183658 3112 consensus_queue.cc:1035] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:27:57.194803 3142 tablet.cc:1722] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:27:57.195442 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_25f0e3650873407a8b3cbd8c8e0c86af"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:57.198460 3091 tablet_replica.cc:330] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:27:57.198549 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet d6cfa2547e6a497a8db35651b03b9995 (table default.Table_35a03bcad08c4020b92156f715f3cdef [id=18c041f9985b4c9f99cbc8cc4f74db94]) successfully deleted
I20260927 23:27:57.199409 3091 raft_consensus.cc:2229] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:57.199898 3091 raft_consensus.cc:2258] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Raft consensus is shut down!
W20260927 23:27:57.198009 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_25f0e3650873407a8b3cbd8c8e0c86af in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:57.210142 3147 raft_consensus.cc:482] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:57.210769 3147 raft_consensus.cc:504] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.214416 3147 leader_election.cc:288] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:57.215474 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a731a36b1da24f559c77bd2b6b484c72" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:27:57.217554 2763 leader_election.cc:341] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:57.218175 3091 ts_tablet_manager.cc:1905] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:57.221328 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a731a36b1da24f559c77bd2b6b484c72" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:27:57.225916 2761 leader_election.cc:341] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:57.226617 2761 leader_election.cc:302] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:57.227605 3147 raft_consensus.cc:2726] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:57.229413 2393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_3597d6af5e24454ab687a0c4be731a86" } modify_external_catalogs: true
I20260927 23:27:57.230074 2393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_3597d6af5e24454ab687a0c4be731a86" } modify_external_catalogs: true
I20260927 23:27:57.251190 3072 tablet_bootstrap.cc:492] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:57.251858 3072 ts_tablet_manager.cc:1397] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.387s user 0.020s sys 0.014s
W20260927 23:27:57.253196 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 75281.1us
I20260927 23:27:57.254576 3072 raft_consensus.cc:348] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:57.255501 3072 raft_consensus.cc:374] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:57.255798 3072 raft_consensus.cc:729] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:57.256641 3072 consensus_queue.cc:260] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:57.258998 3072 ts_tablet_manager.cc:1428] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260927 23:27:57.263722 3072 tablet_bootstrap.cc:492] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:57.273768 2390 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 2572268ca47346b0b78abc6a9ca7f908 (table default.Table_2d889ed8e18e47a79f0541db93242386 [id=103d7c5e210c48fdafb8c199eb4cf676]). Expected version 1 got 0
I20260927 23:27:57.275110 2648 tablet_service.cc:1467] Processing CreateTablet for tablet f5cec41645b24e7d99e3dae3f2e8a22f (DEFAULT_TABLE table=default.Table_bf2e1c7a89464e32b6c8230599344893 [id=60339de81b484f0ba303f5b8cebdaecf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.276539 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5cec41645b24e7d99e3dae3f2e8a22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.278744 2805 tablet_service.cc:1467] Processing CreateTablet for tablet f5cec41645b24e7d99e3dae3f2e8a22f (DEFAULT_TABLE table=default.Table_bf2e1c7a89464e32b6c8230599344893 [id=60339de81b484f0ba303f5b8cebdaecf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.280114 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5cec41645b24e7d99e3dae3f2e8a22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.281221 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 74fc2f03baae4f70b38b8079deb8a854 (DEFAULT_TABLE table=default.Table_a2512a58a2fe47ff928e54a9b0a8f0f1 [id=052cb2b2fbda4be3825be24c2325b85c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.283161 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 74fc2f03baae4f70b38b8079deb8a854 (DEFAULT_TABLE table=default.Table_a2512a58a2fe47ff928e54a9b0a8f0f1 [id=052cb2b2fbda4be3825be24c2325b85c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.285012 2389 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 517d7bfc225a4372bfd79682a550a0d1 (table default.Table_c42131f0bbf640bbbf4b8dc955fcd001 [id=4a83aea286194e6fac09fde39f326c9f]). Expected version 1 got 0
I20260927 23:27:57.285511 2649 tablet_service.cc:1467] Processing CreateTablet for tablet 3c80915269094d3e8554c1655a1c98af (DEFAULT_TABLE table=default.Table_d8ee874db95045efa6c1321ce207e4ea [id=d4bab68cc19145baae5fab5691c88c8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.286170 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74fc2f03baae4f70b38b8079deb8a854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.286124 2807 tablet_service.cc:1467] Processing CreateTablet for tablet 3c80915269094d3e8554c1655a1c98af (DEFAULT_TABLE table=default.Table_d8ee874db95045efa6c1321ce207e4ea [id=d4bab68cc19145baae5fab5691c88c8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.286793 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c80915269094d3e8554c1655a1c98af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.292659 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c80915269094d3e8554c1655a1c98af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.296795 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74fc2f03baae4f70b38b8079deb8a854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.301054 3154 tablet.cc:1722] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:27:57.302273 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet a3a5bae1f0624b16be7940ac9b84c4da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:57.304446 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet d6cfa2547e6a497a8db35651b03b9995 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:57.308251 3092 ts_tablet_manager.cc:1918] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:57.308785 3092 log.cc:1196] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/26d4c74dddca44bdbaac368e8d521fe7
I20260927 23:27:57.309700 3092 ts_tablet_manager.cc:1939] T 26d4c74dddca44bdbaac368e8d521fe7 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:27:57.315584 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 4929fc9e396e48a189c94586d58aa55f (DEFAULT_TABLE table=default.Table_db0f806a353a4dbdb9d64635383bc826 [id=f0d0efa2f91248b784bb2ab87b87fbf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.316258 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 4929fc9e396e48a189c94586d58aa55f (DEFAULT_TABLE table=default.Table_db0f806a353a4dbdb9d64635383bc826 [id=f0d0efa2f91248b784bb2ab87b87fbf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.316866 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4929fc9e396e48a189c94586d58aa55f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.318558 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4929fc9e396e48a189c94586d58aa55f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.333961 3164 raft_consensus.cc:482] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:57.334553 3164 raft_consensus.cc:504] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:27:57.336542 2391 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c80915269094d3e8554c1655a1c98af consensus state has no leader
W20260927 23:27:57.344247 2392 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 43096d258a654532906f59d581f5a5d6 has no consensus state
I20260927 23:27:57.349525 3167 tablet_bootstrap.cc:492] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:57.352563 3115 raft_consensus.cc:482] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:57.353099 3115 raft_consensus.cc:504] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.355446 3115 leader_election.cc:288] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:57.361042 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2572268ca47346b0b78abc6a9ca7f908" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:57.361845 2828 raft_consensus.cc:2454] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:57.363173 2609 leader_election.cc:302] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:27:57.364799 3115 raft_consensus.cc:2781] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:57.365137 3115 raft_consensus.cc:482] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:57.365453 3115 raft_consensus.cc:3037] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:57.368413 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2572268ca47346b0b78abc6a9ca7f908" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:57.370736 3167 tablet_bootstrap.cc:654] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:57.374670 3070 tablet_bootstrap.cc:492] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:57.375221 3070 ts_tablet_manager.cc:1397] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.351s user 0.073s sys 0.087s
I20260927 23:27:57.376181 3115 raft_consensus.cc:504] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.378548 3070 raft_consensus.cc:348] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.379274 3070 raft_consensus.cc:374] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:57.379552 3070 raft_consensus.cc:729] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:57.379382 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2572268ca47346b0b78abc6a9ca7f908" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:57.379981 2828 raft_consensus.cc:3037] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:57.380193 3070 consensus_queue.cc:260] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.382861 3070 ts_tablet_manager.cc:1428] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260927 23:27:57.385221 3070 tablet_bootstrap.cc:492] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:57.387524 3072 tablet_bootstrap.cc:700] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent opening tablet: real 0.119s user 0.004s sys 0.000s
I20260927 23:27:57.388203 3072 tablet_bootstrap.cc:654] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
W20260927 23:27:57.390795 2608 leader_election.cc:341] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:57.391155 3070 tablet_bootstrap.cc:654] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:57.394774 2828 raft_consensus.cc:2454] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:27:57.396101 2609 leader_election.cc:302] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:27:57.396739 3090 ts_tablet_manager.cc:1918] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:27:57.396924 3112 raft_consensus.cc:2781] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:57.397212 3090 log.cc:1196] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/d6cfa2547e6a497a8db35651b03b9995
I20260927 23:27:57.397425 3112 raft_consensus.cc:686] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:27:57.397303 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2572268ca47346b0b78abc6a9ca7f908" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:27:57.398303 3090 ts_tablet_manager.cc:1939] T d6cfa2547e6a497a8db35651b03b9995 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:27:57.398540 3167 tablet_bootstrap.cc:492] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:57.398219 3112 consensus_queue.cc:237] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.398999 3167 ts_tablet_manager.cc:1397] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.050s user 0.008s sys 0.009s
I20260927 23:27:57.403540 3167 raft_consensus.cc:348] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.404361 3167 raft_consensus.cc:374] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:57.404645 3167 raft_consensus.cc:729] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:57.405329 3167 consensus_queue.cc:260] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.407614 3167 ts_tablet_manager.cc:1428] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.008s user 0.002s sys 0.002s
I20260927 23:27:57.407991 3164 leader_election.cc:288] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:27:57.409842 2906 leader_election.cc:341] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Not found: Tablet not found: 26d4c74dddca44bdbaac368e8d521fe7
I20260927 23:27:57.413430 3090 tablet_replica.cc:330] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:27:57.414214 3090 raft_consensus.cc:2229] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:57.414691 3090 raft_consensus.cc:2258] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:57.423035 3090 ts_tablet_manager.cc:1905] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:57.428773 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_27d90c004e8847c69086658e13affacc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:57.430578 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27d90c004e8847c69086658e13affacc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:57.440598 3070 tablet_bootstrap.cc:492] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:57.441084 3070 ts_tablet_manager.cc:1397] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.058s user 0.005s sys 0.008s
I20260927 23:27:57.444533 2393 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 27a659173fcd477583170022cdc766cd
I20260927 23:27:57.444320 3070 raft_consensus.cc:348] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.445082 3070 raft_consensus.cc:374] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:57.445325 3070 raft_consensus.cc:729] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:57.449374 3070 consensus_queue.cc:260] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.457006 3070 ts_tablet_manager.cc:1428] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.016s user 0.008s sys 0.000s
I20260927 23:27:57.457216 3072 tablet_bootstrap.cc:492] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:57.457914 3072 ts_tablet_manager.cc:1397] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.194s user 0.014s sys 0.008s
I20260927 23:27:57.458285 3070 tablet_bootstrap.cc:492] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:57.462186 3072 raft_consensus.cc:348] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.462864 3072 raft_consensus.cc:374] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:57.463107 3072 raft_consensus.cc:729] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:57.463747 3070 tablet_bootstrap.cc:654] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:57.468773 3167 tablet_bootstrap.cc:492] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:57.471586 3090 ts_tablet_manager.cc:1918] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:57.472035 3090 log.cc:1196] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/26d4c74dddca44bdbaac368e8d521fe7
I20260927 23:27:57.472623 3072 consensus_queue.cc:260] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.475490 3167 tablet_bootstrap.cc:654] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:57.478806 2384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d
I20260927 23:27:57.482368 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:57 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:57.485431 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:57 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:57.492228 3072 ts_tablet_manager.cc:1428] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.034s user 0.007s sys 0.000s
I20260927 23:27:57.493044 3072 tablet_bootstrap.cc:492] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:57.495215 3167 tablet_bootstrap.cc:492] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:57.495692 3167 ts_tablet_manager.cc:1397] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.027s user 0.002s sys 0.009s
I20260927 23:27:57.500507 3070 tablet_bootstrap.cc:492] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:57.500953 3070 ts_tablet_manager.cc:1397] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.043s user 0.011s sys 0.000s
I20260927 23:27:57.502674 3091 ts_tablet_manager.cc:1918] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:57.503226 3091 log.cc:1196] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/b5efb2e4760a48c98e1c6e9c1add0eca
I20260927 23:27:57.503463 3070 raft_consensus.cc:348] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.504119 3091 ts_tablet_manager.cc:1939] T b5efb2e4760a48c98e1c6e9c1add0eca P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:27:57.504238 3070 raft_consensus.cc:374] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:57.504552 3070 raft_consensus.cc:729] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:57.505739 3167 raft_consensus.cc:348] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.506481 3167 raft_consensus.cc:374] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:57.506759 3167 raft_consensus.cc:729] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:57.507557 3167 consensus_queue.cc:260] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.509648 3072 tablet_bootstrap.cc:654] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:57.510039 3167 ts_tablet_manager.cc:1428] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.014s user 0.004s sys 0.004s
I20260927 23:27:57.510953 3167 tablet_bootstrap.cc:492] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:57.520478 3090 ts_tablet_manager.cc:1939] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:27:57.527498 3112 raft_consensus.cc:482] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:57.528192 3112 raft_consensus.cc:504] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.529091 3070 consensus_queue.cc:260] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.531153 3070 ts_tablet_manager.cc:1428] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.030s user 0.005s sys 0.000s
I20260927 23:27:57.532035 3070 tablet_bootstrap.cc:492] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:57.532262 3112 leader_election.cc:288] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:57.533245 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60a2bd7ab1bc4fc29cc4e70d6e6de70d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:57.534060 2828 raft_consensus.cc:2454] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:57.535248 2609 leader_election.cc:302] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:27:57.544914 2384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_0a453cae4de74eb2992e691b44d18700" } new_table_name: "default.Table_72f38013370a483784dfabf8e4646d24" modify_external_catalogs: true
I20260927 23:27:57.547077 3091 tablet_replica.cc:330] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:27:57.549674 3167 tablet_bootstrap.cc:654] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:57.550949 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_50e57468f6b742cdab9659eb6efde970"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:57.552747 3070 tablet_bootstrap.cc:654] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:57.548105 3091 raft_consensus.cc:2229] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:27:57.554452 2393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_42b58587cafe46998c02643a817f30be" } new_table_name: "default.Table_d8d7e47a77aa46818639a40ebd279a12" modify_external_catalogs: true
I20260927 23:27:57.555189 3091 pending_rounds.cc:62] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Trying to abort 1 pending ops.
I20260927 23:27:57.555464 3091 pending_rounds.cc:68] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099668219875328 op_type: NO_OP noop_request { }
I20260927 23:27:57.555876 3091 raft_consensus.cc:2866] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:27:57.556245 3091 raft_consensus.cc:2258] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 23:27:57.575256 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_50e57468f6b742cdab9659eb6efde970 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:57.579452 3091 ts_tablet_manager.cc:1905] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:27:57.587172 2384 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 45dd55b5a9d94ff0b84a799174ab5359 has no consensus state
I20260927 23:27:57.590380 3072 tablet_bootstrap.cc:492] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:57.590868 3072 ts_tablet_manager.cc:1397] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.098s user 0.010s sys 0.001s
I20260927 23:27:57.596890 3072 raft_consensus.cc:348] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.597611 3072 raft_consensus.cc:374] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:57.597844 3072 raft_consensus.cc:729] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:57.598526 3072 consensus_queue.cc:260] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:27:57.601102 2378 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.60:41571, user_credentials={real_user=slave}} blocked reactor thread for 328915us
I20260927 23:27:57.604266 2950 tablet_service.cc:1467] Processing CreateTablet for tablet f5cec41645b24e7d99e3dae3f2e8a22f (DEFAULT_TABLE table=default.Table_bf2e1c7a89464e32b6c8230599344893 [id=60339de81b484f0ba303f5b8cebdaecf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.605473 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5cec41645b24e7d99e3dae3f2e8a22f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.620573 3072 ts_tablet_manager.cc:1428] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.029s user 0.002s sys 0.005s
W20260927 23:27:57.625522 2393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6ca42b47d8044adc99a82859bf79e606 has no consensus state
I20260927 23:27:57.628091 3072 tablet_bootstrap.cc:492] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:57.637547 3072 tablet_bootstrap.cc:654] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:57.639645 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_33d39f037aca42d7a1e02a19788cf146"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:57.648631 3147 raft_consensus.cc:482] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:57.649171 3147 raft_consensus.cc:504] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.651151 3147 leader_election.cc:288] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:57.653228 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c80915269094d3e8554c1655a1c98af" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:57.653913 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca42b47d8044adc99a82859bf79e606 (DEFAULT_TABLE table=default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:27:57.654714 2763 leader_election.cc:341] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:57.655104 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca42b47d8044adc99a82859bf79e606. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:27:57.659163 2761 leader_election.cc:341] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: 3c80915269094d3e8554c1655a1c98af
W20260927 23:27:57.659549 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33d39f037aca42d7a1e02a19788cf146 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:57.659664 2761 leader_election.cc:302] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:57.660421 3147 raft_consensus.cc:2726] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:57.652871 2388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_e7672fd6100045e3945ce7ddd03edbf1" } modify_external_catalogs: true
I20260927 23:27:57.666009 2388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_e7672fd6100045e3945ce7ddd03edbf1" } modify_external_catalogs: true
I20260927 23:27:57.668134 3147 raft_consensus.cc:482] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:57.668658 3147 raft_consensus.cc:504] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.672734 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4929fc9e396e48a189c94586d58aa55f" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:57.672657 2807 tablet_service.cc:1467] Processing CreateTablet for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e (DEFAULT_TABLE table=default.Table_1f563f833bd94200a45952661e37e1fe [id=9878c4920bd7428d82faa9bb48c171a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.674083 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:27:57.674187 2763 leader_election.cc:341] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:57.674789 3147 leader_election.cc:288] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:57.675370 3092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 22) took 1154 ms. Trace:
I20260927 23:27:57.675710 3092 rpcz_store.cc:276] 0927 23:27:56.520778 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:56.521879 (+ 1101us) service_pool.cc:227] Handling call
0927 23:27:57.087248 (+565369us) ts_tablet_manager.cc:1145] Deleting tablet 26d4c74dddca44bdbaac368e8d521fe7
0927 23:27:57.087301 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:27:57.145446 (+ 58145us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:57.307961 (+162515us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:57.675299 (+367338us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":564761}
I20260927 23:27:57.677125 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 39b3bf40c51646d38559f6852ebad6f0 (DEFAULT_TABLE table=default.Table_faba7f79481947c1a44139eddeb749a6 [id=2aa2af3d524a455e8d22c0171fe10942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.677563 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 26d4c74dddca44bdbaac368e8d521fe7 (table default.Table_b906f759cf284e0aaf25a13094802a09 [id=dd99f8473f9a4f4f82b2127c1ea61fd2]) successfully deleted
W20260927 23:27:57.677561 2761 leader_election.cc:341] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: 4929fc9e396e48a189c94586d58aa55f
I20260927 23:27:57.678313 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39b3bf40c51646d38559f6852ebad6f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.678221 2761 leader_election.cc:302] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:57.679145 3115 leader_election.cc:288] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:57.679308 3147 raft_consensus.cc:2726] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:57.680017 3115 raft_consensus.cc:2781] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:57.680491 3115 raft_consensus.cc:482] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:57.680831 3115 raft_consensus.cc:3037] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:57.681159 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 45dd55b5a9d94ff0b84a799174ab5359 (DEFAULT_TABLE table=default.Table_72f38013370a483784dfabf8e4646d24 [id=c7a02939c757489cb0b7318f5ed346ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.683156 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45dd55b5a9d94ff0b84a799174ab5359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.684859 3164 raft_consensus.cc:482] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:57.685329 3164 raft_consensus.cc:504] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.687351 3164 leader_election.cc:288] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:57.688897 2386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_27d90c004e8847c69086658e13affacc" } modify_external_catalogs: true
I20260927 23:27:57.689636 2386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_27d90c004e8847c69086658e13affacc" } modify_external_catalogs: true
I20260927 23:27:57.693693 2384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_33057639607344a0bcd2f4850a382324"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:57.696748 2384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_33057639607344a0bcd2f4850a382324 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:57.699221 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2572268ca47346b0b78abc6a9ca7f908" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:57.700027 2828 raft_consensus.cc:2379] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:27:57.700714 2385 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_c42131f0bbf640bbbf4b8dc955fcd001 [id=4a83aea286194e6fac09fde39f326c9f] alter complete (version 1)
I20260927 23:27:57.705677 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 74fc2f03baae4f70b38b8079deb8a854 (DEFAULT_TABLE table=default.Table_a2512a58a2fe47ff928e54a9b0a8f0f1 [id=052cb2b2fbda4be3825be24c2325b85c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.706570 2949 tablet_service.cc:1514] Processing DeleteTablet for tablet af3004c09be14ebdbee12ce87c95e993 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:56 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:57.706858 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74fc2f03baae4f70b38b8079deb8a854. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.708998 2949 tablet_service.cc:1467] Processing CreateTablet for tablet 4929fc9e396e48a189c94586d58aa55f (DEFAULT_TABLE table=default.Table_db0f806a353a4dbdb9d64635383bc826 [id=f0d0efa2f91248b784bb2ab87b87fbf1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.709127 2951 tablet_service.cc:1467] Processing CreateTablet for tablet 3c80915269094d3e8554c1655a1c98af (DEFAULT_TABLE table=default.Table_d8ee874db95045efa6c1321ce207e4ea [id=d4bab68cc19145baae5fab5691c88c8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.710141 2949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4929fc9e396e48a189c94586d58aa55f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.713762 2951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c80915269094d3e8554c1655a1c98af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.718914 2803 tablet_service.cc:1467] Processing CreateTablet for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (DEFAULT_TABLE table=default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.721658 2802 tablet_service.cc:1467] Processing CreateTablet for tablet 43096d258a654532906f59d581f5a5d6 (DEFAULT_TABLE table=default.Table_35e97e4b9b9c4dc49db9e1970f335204 [id=d3de7e8bfbed4409bf10ead542fe1c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.678684 3092 tablet_replica.cc:330] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:27:57.736752 3072 tablet_bootstrap.cc:492] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:57.736871 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:57 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:57.737293 3072 ts_tablet_manager.cc:1397] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.109s user 0.006s sys 0.013s
I20260927 23:27:57.737279 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.739745 3072 raft_consensus.cc:348] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.740682 3072 raft_consensus.cc:374] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:57.740938 3072 raft_consensus.cc:729] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:57.741078 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43096d258a654532906f59d581f5a5d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.741501 3072 consensus_queue.cc:260] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.751042 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 3c80915269094d3e8554c1655a1c98af (table default.Table_44775a3339af4c55ae5f701b0e17b7a1 [id=d4bab68cc19145baae5fab5691c88c8b]). Expected version 1 got 0
I20260927 23:27:57.756582 3092 raft_consensus.cc:2229] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:57.757369 3092 raft_consensus.cc:2258] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:57.760007 3092 ts_tablet_manager.cc:1905] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:57.767447 3072 ts_tablet_manager.cc:1428] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.030s user 0.006s sys 0.000s
I20260927 23:27:57.769734 3167 tablet_bootstrap.cc:492] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:57.772617 3167 ts_tablet_manager.cc:1397] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.262s user 0.023s sys 0.042s
I20260927 23:27:57.775872 3167 raft_consensus.cc:348] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.789040 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet d6cfa2547e6a497a8db35651b03b9995 (table default.Table_35a03bcad08c4020b92156f715f3cdef [id=18c041f9985b4c9f99cbc8cc4f74db94]) successfully deleted
I20260927 23:27:57.789726 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 26d4c74dddca44bdbaac368e8d521fe7 (table default.Table_b906f759cf284e0aaf25a13094802a09 [id=dd99f8473f9a4f4f82b2127c1ea61fd2]) successfully deleted
W20260927 23:27:57.790681 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet af3004c09be14ebdbee12ce87c95e993 because the tablet was not found. No further retry: Not found: Tablet not found: af3004c09be14ebdbee12ce87c95e993
W20260927 23:27:57.791285 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d already in progress: creating tablet
I20260927 23:27:57.794718 2385 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_c42131f0bbf640bbbf4b8dc955fcd001" } new_table_name: "default.Table_29029b98f7f745888cd86bb69c036043" modify_external_catalogs: true
I20260927 23:27:57.794930 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_cbec4626631d4ff983d8ecdb13b43595"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:57.795553 3167 raft_consensus.cc:374] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:57.795900 3167 raft_consensus.cc:729] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
W20260927 23:27:57.796723 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cbec4626631d4ff983d8ecdb13b43595 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:57.799187 3167 consensus_queue.cc:260] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.801535 2388 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet f8be107bd6464e9385bb64408479bbeb
I20260927 23:27:57.811329 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60a2bd7ab1bc4fc29cc4e70d6e6de70d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:57.812875 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 45dd55b5a9d94ff0b84a799174ab5359 (DEFAULT_TABLE table=default.Table_72f38013370a483784dfabf8e4646d24 [id=c7a02939c757489cb0b7318f5ed346ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.814889 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45dd55b5a9d94ff0b84a799174ab5359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.816229 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2572268ca47346b0b78abc6a9ca7f908" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:57.818001 2649 tablet_service.cc:1467] Processing CreateTablet for tablet 39b3bf40c51646d38559f6852ebad6f0 (DEFAULT_TABLE table=default.Table_faba7f79481947c1a44139eddeb749a6 [id=2aa2af3d524a455e8d22c0171fe10942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.819433 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39b3bf40c51646d38559f6852ebad6f0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:27:57.818215 2608 leader_election.cc:341] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:57.821599 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca42b47d8044adc99a82859bf79e606 (DEFAULT_TABLE table=default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.823519 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (DEFAULT_TABLE table=default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.825274 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e (DEFAULT_TABLE table=default.Table_1f563f833bd94200a45952661e37e1fe [id=9878c4920bd7428d82faa9bb48c171a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.826936 2653 tablet_service.cc:1467] Processing CreateTablet for tablet 7096e434ac9f4290ad58ba4f7e7c2513 (DEFAULT_TABLE table=default.Table_3963531cde4641e785245322c0a23fab [id=1cac6a4a98734d4abf99d93891e1f077]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.828639 2654 tablet_service.cc:1467] Processing CreateTablet for tablet 43096d258a654532906f59d581f5a5d6 (DEFAULT_TABLE table=default.Table_35e97e4b9b9c4dc49db9e1970f335204 [id=d3de7e8bfbed4409bf10ead542fe1c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.833422 3163 raft_consensus.cc:482] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:57.834290 3163 raft_consensus.cc:504] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.839946 3070 tablet_bootstrap.cc:492] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:57.840732 3070 ts_tablet_manager.cc:1397] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.309s user 0.020s sys 0.021s
I20260927 23:27:57.840968 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca42b47d8044adc99a82859bf79e606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.842906 3167 ts_tablet_manager.cc:1428] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.070s user 0.007s sys 0.000s
I20260927 23:27:57.843952 3167 tablet_bootstrap.cc:492] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
W20260927 23:27:57.848425 2609 leader_election.cc:341] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Not found: Tablet not found: a3a5bae1f0624b16be7940ac9b84c4da
W20260927 23:27:57.853222 2763 consensus_peers.cc:487] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6cfa2547e6a497a8db35651b03b9995. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:27:57.854140 3112 raft_consensus.cc:482] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:57.854857 3112 raft_consensus.cc:504] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.855639 3070 raft_consensus.cc:348] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.856361 3070 raft_consensus.cc:374] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:57.859225 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a731a36b1da24f559c77bd2b6b484c72" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:57.859879 2828 raft_consensus.cc:2454] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:57.865056 2609 leader_election.cc:302] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:27:57.863061 3163 leader_election.cc:288] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:57.866742 3163 raft_consensus.cc:482] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:57.867269 3163 raft_consensus.cc:504] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.868817 3091 ts_tablet_manager.cc:1918] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:27:57.869311 3091 log.cc:1196] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/a4372ae7a9d6426a97419d89253f99d1
W20260927 23:27:57.843079 2608 leader_election.cc:341] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:27:57.871618 2907 leader_election.cc:341] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 26d4c74dddca44bdbaac368e8d521fe7
I20260927 23:27:57.872138 2907 leader_election.cc:302] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:27:57.872788 2907 raft_consensus.cc:2670] T 26d4c74dddca44bdbaac368e8d521fe7 P de216a4a8e4b4e1b857d8647a3b05f6d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260927 23:27:57.856668 3070 raft_consensus.cc:729] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:57.804594 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet b5efb2e4760a48c98e1c6e9c1add0eca (table default.Table_e3fad1f8c2494735b4723d0ecc1b0b39 [id=601abf06edf443879124d16259163a7e]) successfully deleted
W20260927 23:27:57.874054 2380 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 69480.5us
I20260927 23:27:57.874644 2907 leader_election.cc:302] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:27:57.874822 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet af3004c09be14ebdbee12ce87c95e993 because the tablet was not found. No further retry: Not found: Tablet not found: af3004c09be14ebdbee12ce87c95e993
I20260927 23:27:57.875385 3164 raft_consensus.cc:3037] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:57.843330 2386 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 55effdc150f84fd0a99b80a5d4fce89d
I20260927 23:27:57.847343 2387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_44775a3339af4c55ae5f701b0e17b7a1" } new_table_name: "default.Table_52009c39b67d47ea919822fecb9fcc5f" modify_external_catalogs: true
I20260927 23:27:57.879287 3167 tablet_bootstrap.cc:654] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:57.890328 3164 raft_consensus.cc:2726] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:27:57.862468 2379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6ca42b47d8044adc99a82859bf79e606 consensus state has no leader
I20260927 23:27:57.862084 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_e028dc8270d4406bb436aa0b2caf6dfe"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:57.898665 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e028dc8270d4406bb436aa0b2caf6dfe in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:57.903014 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 7096e434ac9f4290ad58ba4f7e7c2513 (DEFAULT_TABLE table=default.Table_3963531cde4641e785245322c0a23fab [id=1cac6a4a98734d4abf99d93891e1f077]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:27:57.906391 2378 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 115091us
W20260927 23:27:57.910231 2378 consensus_peers.cc:487] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f -> Peer 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571): Couldn't send request to peer 0cdd64d48a684d5ebccc2e2e8a4e3150. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260927 23:27:57.912025 2951 tablet_service.cc:1467] Processing CreateTablet for tablet 6ca42b47d8044adc99a82859bf79e606 (DEFAULT_TABLE table=default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.913261 2951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ca42b47d8044adc99a82859bf79e606. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.917088 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e (DEFAULT_TABLE table=default.Table_1f563f833bd94200a45952661e37e1fe [id=9878c4920bd7428d82faa9bb48c171a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.918159 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.918442 3112 leader_election.cc:288] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:57.919566 3112 raft_consensus.cc:2781] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:57.920027 3112 raft_consensus.cc:482] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:57.920390 3112 raft_consensus.cc:3037] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:57.920408 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 39b3bf40c51646d38559f6852ebad6f0 (DEFAULT_TABLE table=default.Table_faba7f79481947c1a44139eddeb749a6 [id=2aa2af3d524a455e8d22c0171fe10942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.921475 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39b3bf40c51646d38559f6852ebad6f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.921803 3091 ts_tablet_manager.cc:1939] T a4372ae7a9d6426a97419d89253f99d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:27:57.780076 3072 tablet_bootstrap.cc:492] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:57.925179 2947 tablet_service.cc:1467] Processing CreateTablet for tablet 45dd55b5a9d94ff0b84a799174ab5359 (DEFAULT_TABLE table=default.Table_72f38013370a483784dfabf8e4646d24 [id=c7a02939c757489cb0b7318f5ed346ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.926316 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 45dd55b5a9d94ff0b84a799174ab5359. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.927564 3072 tablet_bootstrap.cc:654] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:57.918942 3070 consensus_queue.cc:260] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:57.929131 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (DEFAULT_TABLE table=default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.929975 3070 ts_tablet_manager.cc:1428] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.089s user 0.003s sys 0.003s
I20260927 23:27:57.930224 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.933101 2945 tablet_service.cc:1467] Processing CreateTablet for tablet 43096d258a654532906f59d581f5a5d6 (DEFAULT_TABLE table=default.Table_35e97e4b9b9c4dc49db9e1970f335204 [id=d3de7e8bfbed4409bf10ead542fe1c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.934232 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43096d258a654532906f59d581f5a5d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.937368 2944 tablet_service.cc:1467] Processing CreateTablet for tablet 7096e434ac9f4290ad58ba4f7e7c2513 (DEFAULT_TABLE table=default.Table_3963531cde4641e785245322c0a23fab [id=1cac6a4a98734d4abf99d93891e1f077]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.938534 2944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7096e434ac9f4290ad58ba4f7e7c2513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.961190 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7096e434ac9f4290ad58ba4f7e7c2513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.961942 2654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43096d258a654532906f59d581f5a5d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.964275 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.961066 3163 leader_election.cc:288] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:57.968895 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.974699 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410294c28c49404e9c547b27111712eb" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:57.975068 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3a5bae1f0624b16be7940ac9b84c4da" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:57.975495 2970 raft_consensus.cc:2454] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:57.975582 2971 raft_consensus.cc:2454] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:57.974694 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a731a36b1da24f559c77bd2b6b484c72" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:57.993400 2608 leader_election.cc:302] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:57.994642 2608 leader_election.cc:302] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
W20260927 23:27:57.995476 2608 leader_election.cc:341] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:57.998340 3070 tablet_bootstrap.cc:492] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:57.998492 2647 tablet_service.cc:1467] Processing CreateTablet for tablet fbfa55a66b8042ed8592fbff31e92d55 (DEFAULT_TABLE table=default.Table_50e57468f6b742cdab9659eb6efde970 [id=3b845f646d7f4af99b7889442b4d7d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.999874 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbfa55a66b8042ed8592fbff31e92d55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:57.959580 2803 tablet_service.cc:1467] Processing CreateTablet for tablet 828f870365244777b59e8a0756d7d4c9 (DEFAULT_TABLE table=default.Table_25f0e3650873407a8b3cbd8c8e0c86af [id=48f252a9cdb34eb1b452540f4186285c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:57.942675 3167 tablet_bootstrap.cc:492] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:58.002990 3167 ts_tablet_manager.cc:1397] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.159s user 0.011s sys 0.019s
I20260927 23:27:58.005607 3167 raft_consensus.cc:348] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.006668 3167 raft_consensus.cc:374] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.007159 3167 raft_consensus.cc:729] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:58.008363 3167 consensus_queue.cc:260] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.020093 3163 raft_consensus.cc:2781] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260927 23:27:58.021792 2761 consensus_peers.cc:487] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 -> Peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Couldn't send request to peer de216a4a8e4b4e1b857d8647a3b05f6d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6cfa2547e6a497a8db35651b03b9995. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:27:57.953313 2805 tablet_service.cc:1467] Processing CreateTablet for tablet fbfa55a66b8042ed8592fbff31e92d55 (DEFAULT_TABLE table=default.Table_50e57468f6b742cdab9659eb6efde970 [id=3b845f646d7f4af99b7889442b4d7d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:58.029255 3167 ts_tablet_manager.cc:1428] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.026s user 0.003s sys 0.005s
I20260927 23:27:58.044332 3167 tablet_bootstrap.cc:492] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:57.912544 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7096e434ac9f4290ad58ba4f7e7c2513. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:58.045902 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828f870365244777b59e8a0756d7d4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:58.048637 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbfa55a66b8042ed8592fbff31e92d55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:58.020538 3163 raft_consensus.cc:482] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:58.065436 3091 tablet_replica.cc:330] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:27:58.066047 3163 raft_consensus.cc:3037] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:27:57.982388 2387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3c80915269094d3e8554c1655a1c98af consensus state has no leader
W20260927 23:27:58.084893 2378 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.60:41571, user_credentials={real_user=slave}} blocked reactor thread for 174718us
I20260927 23:27:58.035013 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_7c86d4308c504c4ba092bb6634326544"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:58.084143 2391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_f4808e8ee6d64bbb8e824ed11976279d" } modify_external_catalogs: true
I20260927 23:27:58.085781 2391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_f4808e8ee6d64bbb8e824ed11976279d" } modify_external_catalogs: true
W20260927 23:27:58.086606 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7c86d4308c504c4ba092bb6634326544 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:58.087548 2392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_3963531cde4641e785245322c0a23fab" } modify_external_catalogs: true
I20260927 23:27:58.088073 2392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_3963531cde4641e785245322c0a23fab" } modify_external_catalogs: true
I20260927 23:27:58.071633 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet a4372ae7a9d6426a97419d89253f99d1 (table default.Table_1b6c64cbb7304c48a43dfa5cef404376 [id=5f884e6c6e6445bbbbdeacc26cd347a5]) successfully deleted
W20260927 23:27:58.089823 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 26d4c74dddca44bdbaac368e8d521fe7 because the tablet was not found. No further retry: Not found: Tablet not found: 26d4c74dddca44bdbaac368e8d521fe7
I20260927 23:27:57.983685 3115 raft_consensus.cc:504] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.098191 2947 tablet_service.cc:1467] Processing CreateTablet for tablet fbfa55a66b8042ed8592fbff31e92d55 (DEFAULT_TABLE table=default.Table_50e57468f6b742cdab9659eb6efde970 [id=3b845f646d7f4af99b7889442b4d7d0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:58.105182 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 828f870365244777b59e8a0756d7d4c9 (DEFAULT_TABLE table=default.Table_25f0e3650873407a8b3cbd8c8e0c86af [id=48f252a9cdb34eb1b452540f4186285c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:58.115427 3115 leader_election.cc:288] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:58.118466 3070 tablet_bootstrap.cc:654] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:58.120667 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60a2bd7ab1bc4fc29cc4e70d6e6de70d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:27:58.120788 2384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_89beccb1022745208c07f912ff8d5643"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:58.122553 2384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_89beccb1022745208c07f912ff8d5643 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:58.122761 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828f870365244777b59e8a0756d7d4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:58.129719 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fbfa55a66b8042ed8592fbff31e92d55. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:27:58.129748 2608 leader_election.cc:341] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:58.098275 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 828f870365244777b59e8a0756d7d4c9 (DEFAULT_TABLE table=default.Table_25f0e3650873407a8b3cbd8c8e0c86af [id=48f252a9cdb34eb1b452540f4186285c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:58.134785 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 828f870365244777b59e8a0756d7d4c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:58.139092 3072 tablet_bootstrap.cc:492] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:58.139765 3072 ts_tablet_manager.cc:1397] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.360s user 0.010s sys 0.005s
I20260927 23:27:58.142549 3072 raft_consensus.cc:348] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.143311 3072 raft_consensus.cc:374] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.143558 3072 raft_consensus.cc:729] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:58.148852 3167 tablet_bootstrap.cc:654] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:58.153056 2388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_9fcb69a219404bf4a763d08ffb0a744a" } new_table_name: "default.Table_2cc5b0c1598e4ff282bbf7668ef916f0" modify_external_catalogs: true
I20260927 23:27:58.159797 2387 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 517d7bfc225a4372bfd79682a550a0d1 (table default.Table_29029b98f7f745888cd86bb69c036043 [id=4a83aea286194e6fac09fde39f326c9f]). Expected version 2 got 1
I20260927 23:27:58.160701 3164 raft_consensus.cc:482] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:58.161194 3164 raft_consensus.cc:504] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:27:58.163220 3164 leader_election.cc:288] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:58.163825 3072 consensus_queue.cc:260] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.167680 2387 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 2572268ca47346b0b78abc6a9ca7f908 (table default.Table_2d889ed8e18e47a79f0541db93242386 [id=103d7c5e210c48fdafb8c199eb4cf676]). Expected version 1 got 0
I20260927 23:27:58.167431 3163 raft_consensus.cc:504] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.171118 3163 leader_election.cc:288] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:58.173188 3072 ts_tablet_manager.cc:1428] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.033s user 0.002s sys 0.004s
I20260927 23:27:58.175745 3091 raft_consensus.cc:2229] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:58.179548 3181 raft_consensus.cc:2781] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:58.180042 3181 raft_consensus.cc:482] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:58.180316 3091 raft_consensus.cc:2258] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:58.180445 3181 raft_consensus.cc:3037] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:58.185225 2944 tablet_service.cc:1514] Processing DeleteTablet for tablet a3a5bae1f0624b16be7940ac9b84c4da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:58.186053 3090 tablet_replica.cc:330] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:27:58.186734 3090 raft_consensus.cc:2229] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:58.187192 3090 raft_consensus.cc:2258] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260927 23:27:58.191475 2906 leader_election.cc:341] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Not found: Tablet not found: a4372ae7a9d6426a97419d89253f99d1
I20260927 23:27:58.192276 2951 tablet_service.cc:1514] Processing DeleteTablet for tablet a4372ae7a9d6426a97419d89253f99d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:55 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:58.193892 3090 ts_tablet_manager.cc:1905] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:58.200989 3112 raft_consensus.cc:504] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.206604 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3a5bae1f0624b16be7940ac9b84c4da" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:27:58.211908 3091 ts_tablet_manager.cc:1905] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:27:58.212821 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 347849us
I20260927 23:27:58.214965 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410294c28c49404e9c547b27111712eb" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:58.215729 2828 raft_consensus.cc:2454] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:58.220533 3112 leader_election.cc:288] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:58.221820 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60a2bd7ab1bc4fc29cc4e70d6e6de70d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:58.222536 2828 raft_consensus.cc:3037] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:58.227741 3072 tablet_bootstrap.cc:492] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:58.228892 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a731a36b1da24f559c77bd2b6b484c72" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:27:58.230615 3163 raft_consensus.cc:482] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:27:58.231247 2907 leader_election.cc:341] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: a4372ae7a9d6426a97419d89253f99d1
W20260927 23:27:58.231330 2608 leader_election.cc:341] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: Tablet not RUNNING: STOPPED
I20260927 23:27:58.231076 3163 raft_consensus.cc:504] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.231654 2907 leader_election.cc:302] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:27:58.231943 2608 leader_election.cc:341] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:58.233074 3163 leader_election.cc:288] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:58.233487 3164 raft_consensus.cc:2726] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:58.236785 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a731a36b1da24f559c77bd2b6b484c72" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:58.237444 2827 raft_consensus.cc:3037] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:58.240790 3181 raft_consensus.cc:504] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.241058 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_494317f627504c768872f2b202fd9194"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:58.242888 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_494317f627504c768872f2b202fd9194 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:58.242926 3181 leader_election.cc:288] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:58.244252 2647 tablet_service.cc:1467] Processing CreateTablet for tablet cc1c0da6ea5c4a09a71555f37becae33 (DEFAULT_TABLE table=default.Table_33d39f037aca42d7a1e02a19788cf146 [id=baad95258e1a4358a58abc3fc1cc3737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:58.244975 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5cec41645b24e7d99e3dae3f2e8a22f" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:58.245705 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc1c0da6ea5c4a09a71555f37becae33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:58.245663 2826 raft_consensus.cc:2454] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:58.253356 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5cec41645b24e7d99e3dae3f2e8a22f" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:58.253927 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410294c28c49404e9c547b27111712eb" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:27:58.254540 2971 raft_consensus.cc:3037] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:27:58.255913 2609 leader_election.cc:341] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Not found: Tablet not found: a3a5bae1f0624b16be7940ac9b84c4da
I20260927 23:27:58.256487 2609 leader_election.cc:302] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
W20260927 23:27:58.260859 2609 raft_consensus.cc:2670] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260927 23:27:58.261940 2609 leader_election.cc:302] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:27:58.262032 2971 raft_consensus.cc:2454] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:27:58.262431 3072 tablet_bootstrap.cc:654] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:58.264811 3092 ts_tablet_manager.cc:1918] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:27:58.265223 3092 log.cc:1196] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/af3004c09be14ebdbee12ce87c95e993
I20260927 23:27:58.265892 3092 ts_tablet_manager.cc:1939] T af3004c09be14ebdbee12ce87c95e993 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:27:58.268029 3092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 23) took 1467 ms. Trace:
I20260927 23:27:58.267383 3070 tablet_bootstrap.cc:492] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:58.280687 3070 ts_tablet_manager.cc:1397] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.283s user 0.021s sys 0.006s
I20260927 23:27:58.286068 3112 raft_consensus.cc:2781] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:58.286463 3112 raft_consensus.cc:482] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:58.286796 3112 raft_consensus.cc:3037] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:27:58.287271 2608 leader_election.cc:341] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:58.294167 2608 leader_election.cc:302] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:27:58.295909 3163 raft_consensus.cc:2781] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:58.276197 3090 ts_tablet_manager.cc:1918] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:58.296393 3163 raft_consensus.cc:686] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:27:58.296682 3090 log.cc:1196] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/a3a5bae1f0624b16be7940ac9b84c4da
I20260927 23:27:58.298214 3090 ts_tablet_manager.cc:1939] T a3a5bae1f0624b16be7940ac9b84c4da P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:27:58.297212 3163 consensus_queue.cc:237] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.300902 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet a3a5bae1f0624b16be7940ac9b84c4da (table default.Table_73c1ac5a47a045529546f8ff56205202 [id=5744abf74d4f405d9fc6a7e70804b513]) successfully deleted
I20260927 23:27:58.295720 3181 consensus_queue.cc:1035] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260927 23:27:58.307049 3090 tablet_replica.cc:330] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:27:58.307577 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_67d5684dd07845a3a70af79c67fed142"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:58.309100 3070 raft_consensus.cc:348] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:27:58.309506 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_67d5684dd07845a3a70af79c67fed142 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:58.309860 3070 raft_consensus.cc:374] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.310122 3070 raft_consensus.cc:729] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:58.312749 2390 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_cbec4626631d4ff983d8ecdb13b43595" } new_table_name: "default.Table_746dd85f958f4346bb7c3484f59e75f4" modify_external_catalogs: true
I20260927 23:27:58.316975 3090 raft_consensus.cc:2229] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:58.319481 3090 raft_consensus.cc:2258] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:58.308125 3112 raft_consensus.cc:504] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.310742 3070 consensus_queue.cc:260] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.323019 3112 leader_election.cc:288] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:58.328145 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5cec41645b24e7d99e3dae3f2e8a22f" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:27:58.306102 2945 tablet_service.cc:1467] Processing CreateTablet for tablet cc1c0da6ea5c4a09a71555f37becae33 (DEFAULT_TABLE table=default.Table_33d39f037aca42d7a1e02a19788cf146 [id=baad95258e1a4358a58abc3fc1cc3737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:58.338776 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc1c0da6ea5c4a09a71555f37becae33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:58.269757 2827 raft_consensus.cc:2454] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:27:58.279667 2828 raft_consensus.cc:2454] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:27:58.340765 3092 rpcz_store.cc:276] 0927 23:27:56.801154 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:56.802199 (+ 1045us) service_pool.cc:227] Handling call
0927 23:27:57.678432 (+876233us) ts_tablet_manager.cc:1145] Deleting tablet af3004c09be14ebdbee12ce87c95e993
0927 23:27:57.678478 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:27:58.067720 (+389242us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:58.264540 (+196820us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:58.267955 (+ 3415us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":23206,"tablet-delete.queue_time_us":875544}
I20260927 23:27:58.283100 3167 tablet_bootstrap.cc:492] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:58.297681 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "410294c28c49404e9c547b27111712eb" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:58.324736 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5cec41645b24e7d99e3dae3f2e8a22f" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:58.341997 3167 ts_tablet_manager.cc:1397] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.298s user 0.009s sys 0.006s
I20260927 23:27:58.342237 2822 raft_consensus.cc:3037] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:58.342042 2826 raft_consensus.cc:3037] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:58.345886 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 66f2842218fc42e5b1b83314e5f1a158 (DEFAULT_TABLE table=default.Table_33057639607344a0bcd2f4850a382324 [id=6b1d660c32a74492bb7b47fea2e1132d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:58.347070 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66f2842218fc42e5b1b83314e5f1a158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:58.349921 3090 ts_tablet_manager.cc:1905] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:58.350852 3167 raft_consensus.cc:348] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.351826 3167 raft_consensus.cc:374] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.357285 3072 tablet_bootstrap.cc:492] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:58.352109 3167 raft_consensus.cc:729] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:58.357774 3072 ts_tablet_manager.cc:1397] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.130s user 0.007s sys 0.005s
I20260927 23:27:58.358062 3167 consensus_queue.cc:260] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.362469 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 66f2842218fc42e5b1b83314e5f1a158 (DEFAULT_TABLE table=default.Table_33057639607344a0bcd2f4850a382324 [id=6b1d660c32a74492bb7b47fea2e1132d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:58.363445 3092 tablet_replica.cc:330] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:27:58.364197 3092 raft_consensus.cc:2229] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:27:58.364466 2822 raft_consensus.cc:2454] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:27:58.365327 3092 raft_consensus.cc:2258] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:58.367270 3092 ts_tablet_manager.cc:1905] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:58.368517 2826 raft_consensus.cc:2454] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
W20260927 23:27:58.362166 2608 leader_election.cc:341] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:58.373872 3167 ts_tablet_manager.cc:1428] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.032s user 0.005s sys 0.002s
I20260927 23:27:58.374833 3167 tablet_bootstrap.cc:492] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:58.292150 2803 tablet_service.cc:1467] Processing CreateTablet for tablet cc1c0da6ea5c4a09a71555f37becae33 (DEFAULT_TABLE table=default.Table_33d39f037aca42d7a1e02a19788cf146 [id=baad95258e1a4358a58abc3fc1cc3737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:58.368894 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66f2842218fc42e5b1b83314e5f1a158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:58.378887 3091 ts_tablet_manager.cc:1918] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:58.379333 3091 log.cc:1196] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/a3a5bae1f0624b16be7940ac9b84c4da
I20260927 23:27:58.380012 3091 ts_tablet_manager.cc:1939] T a3a5bae1f0624b16be7940ac9b84c4da P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:27:58.380218 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc1c0da6ea5c4a09a71555f37becae33. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:58.388427 2609 leader_election.cc:302] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:58.389253 3070 ts_tablet_manager.cc:1428] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.108s user 0.004s sys 0.003s
I20260927 23:27:58.389499 3091 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 34) took 1087 ms. Trace:
I20260927 23:27:58.389779 2609 leader_election.cc:302] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:58.390413 3070 tablet_bootstrap.cc:492] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:58.391837 2609 leader_election.cc:302] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:58.392443 3072 raft_consensus.cc:348] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.393136 3072 raft_consensus.cc:374] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.393356 3072 raft_consensus.cc:729] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:58.393976 3072 consensus_queue.cc:260] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.396600 3181 raft_consensus.cc:2781] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:58.397087 3181 raft_consensus.cc:686] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:27:58.399955 3072 ts_tablet_manager.cc:1428] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.042s user 0.006s sys 0.001s
I20260927 23:27:58.401027 3163 raft_consensus.cc:2781] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:58.389886 3091 rpcz_store.cc:276] 0927 23:27:57.301630 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:57.302062 (+ 432us) service_pool.cc:227] Handling call
0927 23:27:58.045435 (+743373us) ts_tablet_manager.cc:1145] Deleting tablet a3a5bae1f0624b16be7940ac9b84c4da
0927 23:27:58.064084 (+ 18649us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:27:58.179986 (+115902us) spinlock_profiling.cc:243] Waited 94.5 ms on lock 0x7b54001301d4. stack: 00007fe1d7372033 00007fe1d7371f30 00007fe1d6ef3534 00000000004db2b4 00000000004db249 00007fe1de27e471 00007fe1de301352 00007fe1de71726c 00007fe1df324cfa 00007fe1df354e10 00007fe1df33be0d 00007fe1df33bdb9 00007fe1df33bd49 00007fe1df33bd11 00007fe1df33af3d 0000000000580784
0927 23:27:58.331804 (+151818us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:58.378643 (+ 46839us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:58.389385 (+ 10742us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":217396608,"tablet-delete.queue_time_us":742682}
I20260927 23:27:58.403605 3115 raft_consensus.cc:2781] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:58.404052 3115 raft_consensus.cc:686] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:27:58.408808 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 66f2842218fc42e5b1b83314e5f1a158 (DEFAULT_TABLE table=default.Table_33057639607344a0bcd2f4850a382324 [id=6b1d660c32a74492bb7b47fea2e1132d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:58.408818 3115 consensus_queue.cc:237] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.410236 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66f2842218fc42e5b1b83314e5f1a158. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:58.404232 3091 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 35) took 1100 ms. Trace:
I20260927 23:27:58.417079 3072 tablet_bootstrap.cc:492] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:58.416955 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet a3a5bae1f0624b16be7940ac9b84c4da (table default.Table_73c1ac5a47a045529546f8ff56205202 [id=5744abf74d4f405d9fc6a7e70804b513]) successfully deleted
I20260927 23:27:58.417076 3091 rpcz_store.cc:276] 0927 23:27:57.304084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:57.304219 (+ 135us) service_pool.cc:227] Handling call
0927 23:27:58.403853 (+1099634us) ts_tablet_manager.cc:1145] Deleting tablet d6cfa2547e6a497a8db35651b03b9995
0927 23:27:58.403924 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:27:58.404174 (+ 250us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1099003}
I20260927 23:27:58.399394 3181 consensus_queue.cc:237] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.425326 3163 raft_consensus.cc:686] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:27:58.426172 3163 consensus_queue.cc:237] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:27:58.433699 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet d6cfa2547e6a497a8db35651b03b9995 because the tablet was not found. No further retry: Not found: Tablet not found: d6cfa2547e6a497a8db35651b03b9995
I20260927 23:27:58.441947 3070 tablet_bootstrap.cc:654] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:58.445672 3091 tablet_replica.cc:330] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:27:58.461783 3072 tablet_bootstrap.cc:654] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:58.464447 3147 raft_consensus.cc:482] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:58.464959 3147 raft_consensus.cc:504] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.467149 3090 ts_tablet_manager.cc:1918] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:58.467463 3147 leader_election.cc:288] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:58.467567 3090 log.cc:1196] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/a4372ae7a9d6426a97419d89253f99d1
I20260927 23:27:58.469882 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet af3004c09be14ebdbee12ce87c95e993 (table default.Table_16715a5a56ca425399cded31536f5133 [id=c1a377dfa52a4e62b53dc8ffbe3ae6dc]) successfully deleted
I20260927 23:27:58.473625 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fc2f03baae4f70b38b8079deb8a854" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:58.475796 2674 raft_consensus.cc:2454] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:27:58.478078 2763 leader_election.cc:302] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:27:58.479015 3147 raft_consensus.cc:2781] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:58.479403 3147 raft_consensus.cc:482] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:58.479715 3147 raft_consensus.cc:3037] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:58.481593 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fc2f03baae4f70b38b8079deb8a854" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:27:58.483525 2761 leader_election.cc:341] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:58.490134 3147 raft_consensus.cc:504] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.504770 3090 ts_tablet_manager.cc:1939] T a4372ae7a9d6426a97419d89253f99d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:27:58.509939 3147 leader_election.cc:288] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:58.509943 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fc2f03baae4f70b38b8079deb8a854" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
W20260927 23:27:58.511338 2761 leader_election.cc:341] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:58.511868 3072 tablet_bootstrap.cc:492] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:58.512403 3072 ts_tablet_manager.cc:1397] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.096s user 0.012s sys 0.000s
I20260927 23:27:58.515785 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet a4372ae7a9d6426a97419d89253f99d1 (table default.Table_1b6c64cbb7304c48a43dfa5cef404376 [id=5f884e6c6e6445bbbbdeacc26cd347a5]) successfully deleted
I20260927 23:27:58.521785 3072 raft_consensus.cc:348] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.522557 3072 raft_consensus.cc:374] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.522804 3072 raft_consensus.cc:729] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:58.472486 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_a5f98162932247a3ae2928ce03a691a4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:58.523434 3072 consensus_queue.cc:260] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:27:58.525174 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a5f98162932247a3ae2928ce03a691a4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:58.539805 3167 tablet_bootstrap.cc:654] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:58.546550 3199 raft_consensus.cc:482] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:58.547101 3199 raft_consensus.cc:504] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.555686 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c80915269094d3e8554c1655a1c98af" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:58.555857 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fc2f03baae4f70b38b8079deb8a854" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:27:58.556437 2674 raft_consensus.cc:3037] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:58.557859 3072 ts_tablet_manager.cc:1428] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.045s user 0.006s sys 0.000s
W20260927 23:27:58.560439 2608 leader_election.cc:341] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:58.561211 3072 tablet_bootstrap.cc:492] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:58.570330 3199 leader_election.cc:288] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:58.572329 3147 raft_consensus.cc:482] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:58.572947 3147 raft_consensus.cc:504] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.575955 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdfe7faa1a4d5d8e340f09f1cae93e" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:27:58.577630 2761 leader_election.cc:341] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:58.580797 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdfe7faa1a4d5d8e340f09f1cae93e" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:27:58.582633 2763 leader_election.cc:341] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:58.583073 2763 leader_election.cc:302] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:58.588171 3147 leader_election.cc:288] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:58.591706 3164 raft_consensus.cc:482] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:58.592176 3164 raft_consensus.cc:504] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:27:58.567425 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 66696.9us
I20260927 23:27:58.594204 3164 leader_election.cc:288] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:58.594940 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c80915269094d3e8554c1655a1c98af" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:58.589318 3147 raft_consensus.cc:2726] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:58.595697 2823 raft_consensus.cc:2454] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:58.597862 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60a2bd7ab1bc4fc29cc4e70d6e6de70d" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:58.599581 2609 leader_election.cc:302] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:58.600986 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60a2bd7ab1bc4fc29cc4e70d6e6de70d" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:58.601337 2674 raft_consensus.cc:2454] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:27:58.601634 3112 raft_consensus.cc:2781] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:58.602638 3112 raft_consensus.cc:482] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:58.602772 2823 raft_consensus.cc:2379] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:27:58.582839 3199 consensus_queue.cc:1035] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:27:58.602967 3112 raft_consensus.cc:3037] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:58.609724 2907 leader_election.cc:302] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:58.611195 3164 raft_consensus.cc:3037] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:58.622377 3072 tablet_bootstrap.cc:654] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:58.623777 3112 raft_consensus.cc:504] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.626873 3164 raft_consensus.cc:2726] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:58.632915 3091 raft_consensus.cc:2229] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:27:58.637550 3070 tablet_bootstrap.cc:492] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:58.637974 3070 ts_tablet_manager.cc:1397] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.248s user 0.008s sys 0.014s
I20260927 23:27:58.638748 3091 pending_rounds.cc:62] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1: Trying to abort 1 pending ops.
I20260927 23:27:58.638988 3091 pending_rounds.cc:68] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099674842042368 op_type: NO_OP noop_request { }
I20260927 23:27:58.639407 3091 raft_consensus.cc:2866] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:27:58.639708 3091 raft_consensus.cc:2258] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:58.642493 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c80915269094d3e8554c1655a1c98af" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:27:58.640230 3070 raft_consensus.cc:348] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.643004 3091 ts_tablet_manager.cc:1905] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:58.643160 3070 raft_consensus.cc:374] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.643429 3070 raft_consensus.cc:729] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
W20260927 23:27:58.644243 2608 leader_election.cc:341] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:58.644024 3070 consensus_queue.cc:260] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.649317 3070 ts_tablet_manager.cc:1428] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260927 23:27:58.650240 3070 tablet_bootstrap.cc:492] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:58.654122 3112 leader_election.cc:288] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:58.656904 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c80915269094d3e8554c1655a1c98af" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:58.657737 2823 raft_consensus.cc:3037] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:27:58.659574 2388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 410294c28c49404e9c547b27111712eb consensus state has no leader
I20260927 23:27:58.665494 2392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 7096e434ac9f4290ad58ba4f7e7c2513
I20260927 23:27:58.669626 2646 tablet_service.cc:1514] Processing DeleteTablet for tablet 7096e434ac9f4290ad58ba4f7e7c2513 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:58.670197 3070 tablet_bootstrap.cc:654] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:58.672014 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_789dc9b24bea430295966df216636877"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:58.674329 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_789dc9b24bea430295966df216636877 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:58.674784 2823 raft_consensus.cc:2454] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:27:58.672009 2384 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 517d7bfc225a4372bfd79682a550a0d1 (table default.Table_29029b98f7f745888cd86bb69c036043 [id=4a83aea286194e6fac09fde39f326c9f]). Expected version 2 got 1
I20260927 23:27:58.677385 2384 catalog_manager.cc:5495] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:27:58.678853 2384 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 2572268ca47346b0b78abc6a9ca7f908 (table default.Table_2d889ed8e18e47a79f0541db93242386 [id=103d7c5e210c48fdafb8c199eb4cf676]). Expected version 1 got 0
I20260927 23:27:58.680363 2391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet a731a36b1da24f559c77bd2b6b484c72
I20260927 23:27:58.684195 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet 7096e434ac9f4290ad58ba4f7e7c2513 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:58.685305 2646 tablet_service.cc:1514] Processing DeleteTablet for tablet a731a36b1da24f559c77bd2b6b484c72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:58.686053 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet a731a36b1da24f559c77bd2b6b484c72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:58.688443 3211 raft_consensus.cc:482] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:58.689706 3092 ts_tablet_manager.cc:1918] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:27:58.690088 3092 log.cc:1196] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/d6cfa2547e6a497a8db35651b03b9995
I20260927 23:27:58.690865 3092 ts_tablet_manager.cc:1939] T d6cfa2547e6a497a8db35651b03b9995 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:27:58.690791 3211 raft_consensus.cc:504] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.695043 3070 tablet_bootstrap.cc:492] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:58.695508 3070 ts_tablet_manager.cc:1397] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.045s user 0.006s sys 0.004s
I20260927 23:27:58.696751 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5cec41645b24e7d99e3dae3f2e8a22f" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:58.700438 3072 tablet_bootstrap.cc:492] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:58.700881 3072 ts_tablet_manager.cc:1397] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.140s user 0.009s sys 0.002s
I20260927 23:27:58.703245 3072 raft_consensus.cc:348] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.703961 3072 raft_consensus.cc:374] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.704473 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5cec41645b24e7d99e3dae3f2e8a22f" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:58.705523 2823 raft_consensus.cc:2379] T f5cec41645b24e7d99e3dae3f2e8a22f P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:27:58.707095 2906 leader_election.cc:302] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:58.708606 3211 leader_election.cc:288] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:58.710451 3211 raft_consensus.cc:3037] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:58.711840 3070 raft_consensus.cc:348] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.712599 3070 raft_consensus.cc:374] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.712853 3070 raft_consensus.cc:729] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
W20260927 23:27:58.712946 2378 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.60:41571, user_credentials={real_user=slave}} blocked reactor thread for 117536us
I20260927 23:27:58.715912 2945 tablet_service.cc:1514] Processing DeleteTablet for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:57 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:58.716583 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 7096e434ac9f4290ad58ba4f7e7c2513 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:58.717181 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet a731a36b1da24f559c77bd2b6b484c72 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:58.717808 3090 tablet_replica.cc:330] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:27:58.725147 3092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 25) took 1726 ms. Trace:
I20260927 23:27:58.725554 3092 rpcz_store.cc:276] 0927 23:27:57.017485 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:57.018601 (+ 1116us) service_pool.cc:227] Handling call
0927 23:27:58.363221 (+1344620us) ts_tablet_manager.cc:1145] Deleting tablet d6cfa2547e6a497a8db35651b03b9995
0927 23:27:58.363324 (+ 103us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:27:58.536271 (+172947us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:58.689439 (+153168us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:58.725064 (+ 35625us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1333765}
I20260927 23:27:58.726923 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet d6cfa2547e6a497a8db35651b03b9995 (table default.Table_35a03bcad08c4020b92156f715f3cdef [id=18c041f9985b4c9f99cbc8cc4f74db94]) successfully deleted
I20260927 23:27:58.728044 3092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 26) took 1709 ms. Trace:
I20260927 23:27:58.728361 3092 rpcz_store.cc:276] 0927 23:27:57.018446 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:57.018873 (+ 427us) service_pool.cc:227] Handling call
0927 23:27:58.727750 (+1708877us) ts_tablet_manager.cc:1145] Deleting tablet b5efb2e4760a48c98e1c6e9c1add0eca
0927 23:27:58.727799 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:27:58.727988 (+ 189us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1697851}
W20260927 23:27:58.729636 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet b5efb2e4760a48c98e1c6e9c1add0eca because the tablet was not found. No further retry: Not found: Tablet not found: b5efb2e4760a48c98e1c6e9c1add0eca
I20260927 23:27:58.731978 2609 leader_election.cc:302] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:58.732933 3163 raft_consensus.cc:2781] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:58.733409 3163 raft_consensus.cc:686] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:27:58.733855 3090 raft_consensus.cc:2229] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:58.734205 3163 consensus_queue.cc:237] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.734822 3090 raft_consensus.cc:2258] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:58.704228 3072 raft_consensus.cc:729] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:58.737154 3090 ts_tablet_manager.cc:1905] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:58.737921 3215 tablet.cc:1722] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:27:58.744874 3092 tablet_replica.cc:330] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:27:58.747702 3092 raft_consensus.cc:2229] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:58.748203 3092 raft_consensus.cc:2258] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:58.744468 3070 consensus_queue.cc:260] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.749956 3092 ts_tablet_manager.cc:1905] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:58.738059 3072 consensus_queue.cc:260] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.754273 3070 ts_tablet_manager.cc:1428] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.058s user 0.009s sys 0.000s
I20260927 23:27:58.755628 3167 tablet_bootstrap.cc:492] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:58.756316 3167 ts_tablet_manager.cc:1397] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.382s user 0.016s sys 0.009s
I20260927 23:27:58.759066 3167 raft_consensus.cc:348] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.759805 3167 raft_consensus.cc:374] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.760538 3216 tablet.cc:1722] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:27:58.763679 3211 raft_consensus.cc:2726] T f5cec41645b24e7d99e3dae3f2e8a22f P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:58.760089 3167 raft_consensus.cc:729] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:58.770917 3070 tablet_bootstrap.cc:492] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:58.763689 3072 ts_tablet_manager.cc:1428] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.063s user 0.006s sys 0.000s
W20260927 23:27:58.774927 2763 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 192271us
I20260927 23:27:58.777171 2763 leader_election.cc:302] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:58.779752 3205 raft_consensus.cc:2781] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:58.780469 3205 raft_consensus.cc:686] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Becoming Leader. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Running, Role: LEADER
I20260927 23:27:58.783960 3205 consensus_queue.cc:237] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.790964 3070 tablet_bootstrap.cc:654] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:58.795013 3072 tablet_bootstrap.cc:492] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:58.806110 3070 tablet_bootstrap.cc:492] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:58.806699 3070 ts_tablet_manager.cc:1397] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.037s user 0.011s sys 0.000s
I20260927 23:27:58.809628 3070 raft_consensus.cc:348] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.811161 3070 raft_consensus.cc:374] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.811419 3070 raft_consensus.cc:729] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
W20260927 23:27:58.818513 2390 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9cce47b050b445dfba7be5ea3b6a8636 has no consensus state
I20260927 23:27:58.833153 3070 consensus_queue.cc:260] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.843959 3070 ts_tablet_manager.cc:1428] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.037s user 0.007s sys 0.000s
I20260927 23:27:58.846143 3070 tablet_bootstrap.cc:492] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:58.851755 3070 tablet_bootstrap.cc:654] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:58.860450 3072 tablet_bootstrap.cc:654] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:58.867038 2387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_494317f627504c768872f2b202fd9194" } new_table_name: "default.Table_485140265fb4433280c5e7ae5316f122" modify_external_catalogs: true
W20260927 23:27:58.870537 2380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 9cce47b050b445dfba7be5ea3b6a8636 (table default.Table_746dd85f958f4346bb7c3484f59e75f4 [id=b12aae65f933476ab57974d4b328a266], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9cce47b050b445dfba7be5ea3b6a8636 has no consensus state
I20260927 23:27:58.870601 2390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_33057639607344a0bcd2f4850a382324" } modify_external_catalogs: true
I20260927 23:27:58.871145 2390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_33057639607344a0bcd2f4850a382324" } modify_external_catalogs: true
I20260927 23:27:58.792395 3218 tablet.cc:1722] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:27:58.764935 3167 consensus_queue.cc:260] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.883466 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_0af4c8ade9d14c7ba8662aa76259b23c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:58.884678 3167 ts_tablet_manager.cc:1428] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.128s user 0.002s sys 0.004s
W20260927 23:27:58.885455 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0af4c8ade9d14c7ba8662aa76259b23c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:58.885737 3167 tablet_bootstrap.cc:492] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:27:58.887578 2393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_29029b98f7f745888cd86bb69c036043" } modify_external_catalogs: true
I20260927 23:27:58.888156 2393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_29029b98f7f745888cd86bb69c036043" } modify_external_catalogs: true
I20260927 23:27:58.889022 3070 tablet_bootstrap.cc:492] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:58.889436 3070 ts_tablet_manager.cc:1397] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.045s user 0.011s sys 0.000s
I20260927 23:27:58.891805 3070 raft_consensus.cc:348] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.892560 3070 raft_consensus.cc:374] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.892803 3070 raft_consensus.cc:729] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:58.896575 3070 consensus_queue.cc:260] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.899479 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_5f156a98d2694e9aa22d5bcce0e0890d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:58.901266 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5f156a98d2694e9aa22d5bcce0e0890d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:58.907189 3070 ts_tablet_manager.cc:1428] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260927 23:27:58.914472 3072 tablet_bootstrap.cc:492] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:58.914932 3072 ts_tablet_manager.cc:1397] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.120s user 0.001s sys 0.009s
I20260927 23:27:58.930130 3070 tablet_bootstrap.cc:492] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:58.934834 3072 raft_consensus.cc:348] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.935782 3072 raft_consensus.cc:374] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:58.936025 3072 raft_consensus.cc:729] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:58.937731 3070 tablet_bootstrap.cc:654] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:58.940580 3092 ts_tablet_manager.cc:1918] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:58.941457 3072 consensus_queue.cc:260] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:58.944515 3092 log.cc:1196] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/60a2bd7ab1bc4fc29cc4e70d6e6de70d
I20260927 23:27:58.945348 3092 ts_tablet_manager.cc:1939] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:27:58.962478 3090 ts_tablet_manager.cc:1918] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:58.962986 3090 log.cc:1196] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/60a2bd7ab1bc4fc29cc4e70d6e6de70d
I20260927 23:27:58.963626 3090 ts_tablet_manager.cc:1939] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:27:58.964278 3072 ts_tablet_manager.cc:1428] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.049s user 0.009s sys 0.000s
I20260927 23:27:58.968763 3092 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 32) took 1483 ms. Trace:
I20260927 23:27:58.975461 3092 rpcz_store.cc:276] 0927 23:27:57.485103 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:57.485276 (+ 173us) service_pool.cc:227] Handling call
0927 23:27:58.744680 (+1259404us) ts_tablet_manager.cc:1145] Deleting tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d
0927 23:27:58.744732 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:27:58.876814 (+132082us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:58.940228 (+ 63414us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:58.968681 (+ 28453us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1258789}
I20260927 23:27:58.977313 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d (table default.Table_5f321d8241ab4a69a82e325e59605ab2 [id=c227fcbf71824cb49a4c3061117fed02]) successfully deleted
W20260927 23:27:58.978191 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 7096e434ac9f4290ad58ba4f7e7c2513 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7096e434ac9f4290ad58ba4f7e7c2513 already in progress: creating tablet
I20260927 23:27:58.978451 3092 tablet_replica.cc:330] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:27:58.985322 3092 raft_consensus.cc:2229] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:58.995503 3092 raft_consensus.cc:2258] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:58.997763 3092 ts_tablet_manager.cc:1905] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:59.008342 3211 raft_consensus.cc:482] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.008962 3211 raft_consensus.cc:504] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.011155 3211 leader_election.cc:288] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.013146 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a731a36b1da24f559c77bd2b6b484c72" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:27:59.015244 2906 leader_election.cc:341] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: Tablet not RUNNING: STOPPED
I20260927 23:27:59.016173 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a731a36b1da24f559c77bd2b6b484c72" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:59.018129 2907 leader_election.cc:302] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:59.019644 3211 raft_consensus.cc:2726] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:59.020844 3070 tablet_bootstrap.cc:492] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:59.021319 3070 ts_tablet_manager.cc:1397] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.092s user 0.018s sys 0.009s
I20260927 23:27:59.023383 2388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_35e97e4b9b9c4dc49db9e1970f335204" } new_table_name: "default.Table_3777b02a50e04654b10cc6bd6c73f51e" modify_external_catalogs: true
I20260927 23:27:59.023792 3070 raft_consensus.cc:348] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.032663 3070 raft_consensus.cc:374] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.033003 3070 raft_consensus.cc:729] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.033963 3070 consensus_queue.cc:260] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.042155 3070 ts_tablet_manager.cc:1428] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260927 23:27:59.043081 3070 tablet_bootstrap.cc:492] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:59.059494 3070 tablet_bootstrap.cc:654] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.063580 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_c632a6c33818436ca316f423eb2b3aa7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:59.066028 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c632a6c33818436ca316f423eb2b3aa7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:59.070483 3072 tablet_bootstrap.cc:492] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:58.990069 3091 ts_tablet_manager.cc:1918] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:27:59.071761 3091 log.cc:1196] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/60a2bd7ab1bc4fc29cc4e70d6e6de70d
I20260927 23:27:59.072588 3091 ts_tablet_manager.cc:1939] T 60a2bd7ab1bc4fc29cc4e70d6e6de70d P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:27:59.019682 3167 tablet_bootstrap.cc:654] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.079949 3072 tablet_bootstrap.cc:654] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
W20260927 23:27:59.092180 2387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e104d49543934e5db3b188e1db065cdc has no consensus state
I20260927 23:27:59.096055 3091 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 37) took 1615 ms. Trace:
I20260927 23:27:59.096472 3091 rpcz_store.cc:276] 0927 23:27:57.480725 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:57.482198 (+ 1473us) service_pool.cc:227] Handling call
0927 23:27:58.422147 (+939949us) ts_tablet_manager.cc:1145] Deleting tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d
0927 23:27:58.445540 (+ 23393us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:27:58.638257 (+192717us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54001306d4. stack: 00007fe1d7372033 00007fe1d7371f30 00007fe1d6ef3534 00000000004db2b4 00000000004db249 00007fe1de27e471 00007fe1de301352 00007fe1de71726c 00007fe1df324cfa 00007fe1df354e10 00007fe1df33be0d 00007fe1df33bdb9 00007fe1df33bd49 00007fe1df33bd11 00007fe1df33af3d 0000000000580784
0927 23:27:58.899034 (+260777us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:58.989789 (+ 90755us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:59.095964 (+106175us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":428782848,"tablet-delete.queue_time_us":939296}
I20260927 23:27:59.100484 3090 tablet_replica.cc:330] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:27:59.101353 3090 raft_consensus.cc:2229] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:59.096066 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d (table default.Table_5f321d8241ab4a69a82e325e59605ab2 [id=c227fcbf71824cb49a4c3061117fed02]) successfully deleted
W20260927 23:27:59.104983 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 7096e434ac9f4290ad58ba4f7e7c2513 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7096e434ac9f4290ad58ba4f7e7c2513 already in progress: creating tablet
I20260927 23:27:59.098078 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d (table default.Table_5f321d8241ab4a69a82e325e59605ab2 [id=c227fcbf71824cb49a4c3061117fed02]) successfully deleted
I20260927 23:27:59.118813 3072 tablet_bootstrap.cc:492] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:59.135452 3072 ts_tablet_manager.cc:1397] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.065s user 0.011s sys 0.000s
I20260927 23:27:59.135777 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_a902e331f690448a872f6c104857f2ca"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:59.137683 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a902e331f690448a872f6c104857f2ca in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:59.137907 3072 raft_consensus.cc:348] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.138566 3072 raft_consensus.cc:374] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.138783 3072 raft_consensus.cc:729] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.139878 3072 consensus_queue.cc:260] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.101866 3090 raft_consensus.cc:2258] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:59.144520 3211 raft_consensus.cc:482] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.149885 3211 raft_consensus.cc:504] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.151422 3072 ts_tablet_manager.cc:1428] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260927 23:27:59.152005 3211 leader_election.cc:288] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.154330 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c80915269094d3e8554c1655a1c98af" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:59.155886 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c80915269094d3e8554c1655a1c98af" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:59.156775 2823 raft_consensus.cc:2379] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:27:59.158898 3072 tablet_bootstrap.cc:492] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:59.160921 2906 leader_election.cc:302] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:59.165019 3072 tablet_bootstrap.cc:654] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.169689 3211 raft_consensus.cc:3037] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:59.173218 3070 tablet_bootstrap.cc:492] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:59.173785 3070 ts_tablet_manager.cc:1397] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.131s user 0.020s sys 0.029s
I20260927 23:27:59.176403 3070 raft_consensus.cc:348] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.177120 3070 raft_consensus.cc:374] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.177371 3070 raft_consensus.cc:729] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.178232 3072 tablet_bootstrap.cc:492] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:59.178000 3070 consensus_queue.cc:260] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.178668 3072 ts_tablet_manager.cc:1397] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.020s user 0.015s sys 0.002s
I20260927 23:27:59.180986 3072 raft_consensus.cc:348] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.181639 3072 raft_consensus.cc:374] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.182099 3072 raft_consensus.cc:729] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.182675 3072 consensus_queue.cc:260] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.187763 3070 ts_tablet_manager.cc:1428] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260927 23:27:59.188771 3070 tablet_bootstrap.cc:492] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:59.190938 3072 ts_tablet_manager.cc:1428] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.012s user 0.006s sys 0.001s
I20260927 23:27:59.191267 3092 ts_tablet_manager.cc:1918] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:59.192273 3072 tablet_bootstrap.cc:492] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:59.194983 3163 consensus_queue.cc:1035] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:27:59.198827 3072 tablet_bootstrap.cc:654] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.200241 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 2572268ca47346b0b78abc6a9ca7f908 (table default.Table_2d889ed8e18e47a79f0541db93242386 [id=103d7c5e210c48fdafb8c199eb4cf676]). Expected version 1 got 0
I20260927 23:27:59.206971 2385 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 2572268ca47346b0b78abc6a9ca7f908 (table default.Table_2d889ed8e18e47a79f0541db93242386 [id=103d7c5e210c48fdafb8c199eb4cf676]). Expected version 1 got 0
I20260927 23:27:59.212375 3070 tablet_bootstrap.cc:654] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.219316 2803 tablet_service.cc:1467] Processing CreateTablet for tablet 9670df27c4404acda97661c69420b5cf (DEFAULT_TABLE table=default.Table_89beccb1022745208c07f912ff8d5643 [id=1786524f92bb47f4b74e15df9bc77e98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.221148 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 5efbd0211cf3408cb5ea9640d8da6694 (DEFAULT_TABLE table=default.Table_7c86d4308c504c4ba092bb6634326544 [id=0aa0174883974249b1120d0f02be108d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.222512 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 1d299ca52cfc493499b5eaf0acdeb5f9 (DEFAULT_TABLE table=default.Table_e028dc8270d4406bb436aa0b2caf6dfe [id=1af28dc40a8d408ba31c695e8563efc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.222929 3167 tablet_bootstrap.cc:492] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:27:59.223367 3167 ts_tablet_manager.cc:1397] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.338s user 0.013s sys 0.000s
I20260927 23:27:59.225050 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 9670df27c4404acda97661c69420b5cf (DEFAULT_TABLE table=default.Table_89beccb1022745208c07f912ff8d5643 [id=1786524f92bb47f4b74e15df9bc77e98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.225684 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 1d299ca52cfc493499b5eaf0acdeb5f9 (DEFAULT_TABLE table=default.Table_e028dc8270d4406bb436aa0b2caf6dfe [id=1af28dc40a8d408ba31c695e8563efc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.226235 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9670df27c4404acda97661c69420b5cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.226943 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d299ca52cfc493499b5eaf0acdeb5f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.229359 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9670df27c4404acda97661c69420b5cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.234025 3072 tablet_bootstrap.cc:492] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:59.234498 3072 ts_tablet_manager.cc:1397] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.042s user 0.005s sys 0.009s
I20260927 23:27:59.234444 2947 tablet_service.cc:1467] Processing CreateTablet for tablet 9670df27c4404acda97661c69420b5cf (DEFAULT_TABLE table=default.Table_89beccb1022745208c07f912ff8d5643 [id=1786524f92bb47f4b74e15df9bc77e98]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.235721 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9670df27c4404acda97661c69420b5cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.237237 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 5efbd0211cf3408cb5ea9640d8da6694 (DEFAULT_TABLE table=default.Table_7c86d4308c504c4ba092bb6634326544 [id=0aa0174883974249b1120d0f02be108d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.237677 3072 raft_consensus.cc:348] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.238344 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5efbd0211cf3408cb5ea9640d8da6694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.238371 3072 raft_consensus.cc:374] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.238765 3072 raft_consensus.cc:729] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.238740 2384 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 2572268ca47346b0b78abc6a9ca7f908 (table default.Table_2d889ed8e18e47a79f0541db93242386 [id=103d7c5e210c48fdafb8c199eb4cf676]). Expected version 1 got 0
I20260927 23:27:59.239341 3072 consensus_queue.cc:260] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.239952 2384 catalog_manager.cc:5495] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:27:59.241276 2945 tablet_service.cc:1467] Processing CreateTablet for tablet 1d299ca52cfc493499b5eaf0acdeb5f9 (DEFAULT_TABLE table=default.Table_e028dc8270d4406bb436aa0b2caf6dfe [id=1af28dc40a8d408ba31c695e8563efc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.241436 2384 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 410294c28c49404e9c547b27111712eb (table default.Table_2cc5b0c1598e4ff282bbf7668ef916f0 [id=b215370f7b594ab28c3e2a34de2a6f16]). Expected version 1 got 0
I20260927 23:27:59.241748 3072 ts_tablet_manager.cc:1428] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260927 23:27:59.242470 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d299ca52cfc493499b5eaf0acdeb5f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.242424 2384 catalog_manager.cc:5495] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:27:59.243830 2384 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 3c80915269094d3e8554c1655a1c98af (table default.Table_52009c39b67d47ea919822fecb9fcc5f [id=d4bab68cc19145baae5fab5691c88c8b]). Expected version 2 got 0
I20260927 23:27:59.227888 2647 tablet_service.cc:1467] Processing CreateTablet for tablet 5efbd0211cf3408cb5ea9640d8da6694 (DEFAULT_TABLE table=default.Table_7c86d4308c504c4ba092bb6634326544 [id=0aa0174883974249b1120d0f02be108d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.245457 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5efbd0211cf3408cb5ea9640d8da6694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.257840 2384 catalog_manager.cc:5495] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:27:59.260893 2393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 517d7bfc225a4372bfd79682a550a0d1
I20260927 23:27:59.264168 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 517d7bfc225a4372bfd79682a550a0d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:59.266185 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet 517d7bfc225a4372bfd79682a550a0d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:59.276357 3092 log.cc:1196] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/a731a36b1da24f559c77bd2b6b484c72
I20260927 23:27:59.277405 3092 ts_tablet_manager.cc:1939] T a731a36b1da24f559c77bd2b6b484c72 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
W20260927 23:27:59.298928 2388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 43096d258a654532906f59d581f5a5d6 consensus state has no leader
I20260927 23:27:59.301641 3090 ts_tablet_manager.cc:1905] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:59.302172 3091 tablet_replica.cc:330] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:27:59.302933 3091 raft_consensus.cc:2229] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:59.303391 3091 raft_consensus.cc:2258] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:59.305123 2393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_89beccb1022745208c07f912ff8d5643" } new_table_name: "default.Table_fe4babd936e548c7906748e495d790a1" modify_external_catalogs: true
I20260927 23:27:59.311779 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 9cce47b050b445dfba7be5ea3b6a8636 (DEFAULT_TABLE table=default.Table_746dd85f958f4346bb7c3484f59e75f4 [id=b12aae65f933476ab57974d4b328a266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.312439 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_0c873b78624f41d18d0d82e10a46d738"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:59.313004 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cce47b050b445dfba7be5ea3b6a8636. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:27:59.314323 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0c873b78624f41d18d0d82e10a46d738 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:59.319474 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 9cce47b050b445dfba7be5ea3b6a8636 (DEFAULT_TABLE table=default.Table_746dd85f958f4346bb7c3484f59e75f4 [id=b12aae65f933476ab57974d4b328a266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.320817 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cce47b050b445dfba7be5ea3b6a8636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.322939 3164 raft_consensus.cc:482] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.323417 3164 raft_consensus.cc:504] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.324328 3211 raft_consensus.cc:2726] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:59.327659 3072 tablet_bootstrap.cc:492] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:59.341459 3091 ts_tablet_manager.cc:1905] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:59.344496 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45dd55b5a9d94ff0b84a799174ab5359" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:27:59.346875 2907 leader_election.cc:341] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:59.347584 3164 leader_election.cc:288] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.356576 3070 tablet_bootstrap.cc:492] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:59.357091 3070 ts_tablet_manager.cc:1397] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.169s user 0.036s sys 0.032s
I20260927 23:27:59.367286 3070 raft_consensus.cc:348] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.368028 3070 raft_consensus.cc:374] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.368271 3070 raft_consensus.cc:729] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.369019 3070 consensus_queue.cc:260] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.372841 3070 ts_tablet_manager.cc:1428] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.015s user 0.008s sys 0.001s
I20260927 23:27:59.373780 3070 tablet_bootstrap.cc:492] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:59.379815 3072 tablet_bootstrap.cc:654] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.385650 3070 tablet_bootstrap.cc:654] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.404206 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_f21bcb78bde641968c28039a3bf9c800"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:59.406009 3070 tablet_bootstrap.cc:492] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:59.406456 3070 ts_tablet_manager.cc:1397] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.033s user 0.009s sys 0.007s
W20260927 23:27:59.407454 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f21bcb78bde641968c28039a3bf9c800 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:59.409053 3070 raft_consensus.cc:348] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.409873 3070 raft_consensus.cc:374] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.410277 3070 raft_consensus.cc:729] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.410912 3070 consensus_queue.cc:260] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.413913 3196 raft_consensus.cc:482] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.414335 3196 raft_consensus.cc:504] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.416174 3196 leader_election.cc:288] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.418113 3070 ts_tablet_manager.cc:1428] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.011s user 0.006s sys 0.001s
I20260927 23:27:59.420239 3070 tablet_bootstrap.cc:492] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:59.422904 3072 tablet_bootstrap.cc:492] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:59.423401 3072 ts_tablet_manager.cc:1397] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.096s user 0.006s sys 0.003s
I20260927 23:27:59.425848 3070 tablet_bootstrap.cc:654] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.425886 3072 raft_consensus.cc:348] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.426596 3072 raft_consensus.cc:374] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.426923 3072 raft_consensus.cc:729] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.439996 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43096d258a654532906f59d581f5a5d6" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:27:59.443737 2608 leader_election.cc:341] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:59.427713 3072 consensus_queue.cc:260] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:27:59.457181 2393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9670df27c4404acda97661c69420b5cf consensus state has no leader
I20260927 23:27:59.458784 3072 ts_tablet_manager.cc:1428] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.035s user 0.009s sys 0.000s
I20260927 23:27:59.459012 2390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 66f2842218fc42e5b1b83314e5f1a158
I20260927 23:27:59.463109 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 66f2842218fc42e5b1b83314e5f1a158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:59.462477 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet 66f2842218fc42e5b1b83314e5f1a158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:27:59.465270 3072 tablet_bootstrap.cc:492] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:59.467520 3091 ts_tablet_manager.cc:1918] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:27:59.468083 3091 log.cc:1196] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/7096e434ac9f4290ad58ba4f7e7c2513
I20260927 23:27:59.468942 3091 ts_tablet_manager.cc:1939] T 7096e434ac9f4290ad58ba4f7e7c2513 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:27:59.469827 3112 raft_consensus.cc:482] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.470295 3112 raft_consensus.cc:504] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.473075 3164 raft_consensus.cc:482] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.473575 3164 raft_consensus.cc:504] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.475519 3164 leader_election.cc:288] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.477447 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4929fc9e396e48a189c94586d58aa55f" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:59.478292 2673 raft_consensus.cc:2454] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:27:59.479738 2907 leader_election.cc:302] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:27:59.480904 3112 leader_election.cc:288] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.483259 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 7096e434ac9f4290ad58ba4f7e7c2513 (table default.Table_3963531cde4641e785245322c0a23fab [id=1cac6a4a98734d4abf99d93891e1f077]) successfully deleted
I20260927 23:27:59.481633 3164 raft_consensus.cc:2781] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:59.484402 3091 tablet_replica.cc:330] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:27:59.484598 3164 raft_consensus.cc:482] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:59.485474 3091 raft_consensus.cc:2229] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:27:59.484952 3164 raft_consensus.cc:3037] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:59.486457 3091 pending_rounds.cc:62] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1: Trying to abort 1 pending ops.
I20260927 23:27:59.491282 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4929fc9e396e48a189c94586d58aa55f" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:59.489663 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:57 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:59.493084 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 7096e434ac9f4290ad58ba4f7e7c2513 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:27:59.492470 3091 pending_rounds.cc:68] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099674825834496 op_type: NO_OP noop_request { }
I20260927 23:27:59.493902 3091 raft_consensus.cc:2866] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:27:59.494365 3091 raft_consensus.cc:2258] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:59.494722 3211 raft_consensus.cc:482] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.495441 3072 tablet_bootstrap.cc:654] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.498478 3091 ts_tablet_manager.cc:1905] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:59.495229 3211 raft_consensus.cc:504] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.500432 3211 leader_election.cc:288] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.510527 3112 raft_consensus.cc:482] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.504771 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95216fdc2e104a2e9fc4bc424bdbe5e9" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:59.511019 3112 raft_consensus.cc:504] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.513833 2673 raft_consensus.cc:2454] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:27:59.514869 3112 leader_election.cc:288] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.514985 2390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_2d889ed8e18e47a79f0541db93242386" } modify_external_catalogs: true
I20260927 23:27:59.515703 2390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_2d889ed8e18e47a79f0541db93242386" } modify_external_catalogs: true
I20260927 23:27:59.517102 3112 consensus_queue.cc:1035] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:27:59.517683 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95216fdc2e104a2e9fc4bc424bdbe5e9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:59.518378 2972 raft_consensus.cc:2454] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:59.519491 2608 leader_election.cc:302] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:27:59.520937 2907 leader_election.cc:302] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:27:59.521098 3112 raft_consensus.cc:2781] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:59.521618 3112 raft_consensus.cc:482] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:59.521713 3211 raft_consensus.cc:2781] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:59.521944 3112 raft_consensus.cc:3037] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:59.522042 3211 raft_consensus.cc:482] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:59.522374 3211 raft_consensus.cc:3037] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:59.530273 3211 raft_consensus.cc:504] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.533830 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95216fdc2e104a2e9fc4bc424bdbe5e9" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:27:59.536917 3211 leader_election.cc:288] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.540570 2384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_2cc5b0c1598e4ff282bbf7668ef916f0" } new_table_name: "default.Table_813e5d484f8f462b86e9136a57eebacd" modify_external_catalogs: true
I20260927 23:27:59.556648 3072 tablet_bootstrap.cc:492] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:59.557317 3072 ts_tablet_manager.cc:1397] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.092s user 0.014s sys 0.000s
I20260927 23:27:59.560041 3072 raft_consensus.cc:348] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.560851 3072 raft_consensus.cc:374] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.561102 3072 raft_consensus.cc:729] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.561756 3072 consensus_queue.cc:260] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.564852 3072 ts_tablet_manager.cc:1428] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260927 23:27:59.567728 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d299ca52cfc493499b5eaf0acdeb5f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.568516 3072 tablet_bootstrap.cc:492] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:59.572886 2802 tablet_service.cc:1514] Processing DeleteTablet for tablet 517d7bfc225a4372bfd79682a550a0d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:59.573911 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 9cce47b050b445dfba7be5ea3b6a8636 (DEFAULT_TABLE table=default.Table_746dd85f958f4346bb7c3484f59e75f4 [id=b12aae65f933476ab57974d4b328a266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.575095 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cce47b050b445dfba7be5ea3b6a8636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.575273 3072 tablet_bootstrap.cc:654] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.575434 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4929fc9e396e48a189c94586d58aa55f" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:59.576082 2826 raft_consensus.cc:2454] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:27:59.576547 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet 66f2842218fc42e5b1b83314e5f1a158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:27:59.577028 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95216fdc2e104a2e9fc4bc424bdbe5e9" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:59.575124 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45dd55b5a9d94ff0b84a799174ab5359" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:59.581954 3091 ts_tablet_manager.cc:1918] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:27:59.582397 3091 log.cc:1196] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/a731a36b1da24f559c77bd2b6b484c72
I20260927 23:27:59.583531 3091 ts_tablet_manager.cc:1939] T a731a36b1da24f559c77bd2b6b484c72 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:27:59.587437 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet a731a36b1da24f559c77bd2b6b484c72 (table default.Table_f4808e8ee6d64bbb8e824ed11976279d [id=94bf26c26f974c258ee084dbc165be72]) successfully deleted
I20260927 23:27:59.589004 3092 tablet_replica.cc:330] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:27:59.589771 3092 raft_consensus.cc:2229] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:27:59.589915 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43096d258a654532906f59d581f5a5d6" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:59.590278 3092 raft_consensus.cc:2258] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:59.592562 3091 tablet_replica.cc:330] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:27:59.593407 3091 raft_consensus.cc:2229] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:27:59.594312 3091 raft_consensus.cc:2258] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:27:59.597108 3072 tablet_bootstrap.cc:492] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:59.597621 3072 ts_tablet_manager.cc:1397] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.029s user 0.007s sys 0.002s
I20260927 23:27:59.599975 3072 raft_consensus.cc:348] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.600765 3072 raft_consensus.cc:374] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.601032 3072 raft_consensus.cc:729] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.603216 3091 ts_tablet_manager.cc:1905] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:59.606329 3072 consensus_queue.cc:260] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.609622 3072 ts_tablet_manager.cc:1428] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260927 23:27:59.618415 3072 tablet_bootstrap.cc:492] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:59.619016 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_6568f80896124508811607305bf18ff9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:27:59.621008 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6568f80896124508811607305bf18ff9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:59.627322 3072 tablet_bootstrap.cc:654] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.630578 2647 tablet_service.cc:1467] Processing CreateTablet for tablet fac49c5b79e347158621dda52f79e912 (DEFAULT_TABLE table=default.Table_0af4c8ade9d14c7ba8662aa76259b23c [id=efae9c5737164e9d811365b2f6763bb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.632011 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fac49c5b79e347158621dda52f79e912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.639438 2385 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 2572268ca47346b0b78abc6a9ca7f908 (table default.Table_2d889ed8e18e47a79f0541db93242386 [id=103d7c5e210c48fdafb8c199eb4cf676]). Expected version 1 got 0
I20260927 23:27:59.640499 2385 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 410294c28c49404e9c547b27111712eb (table default.Table_2cc5b0c1598e4ff282bbf7668ef916f0 [id=b215370f7b594ab28c3e2a34de2a6f16]). Expected version 1 got 0
I20260927 23:27:59.640605 3196 raft_consensus.cc:482] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.641144 3196 raft_consensus.cc:504] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.643332 3196 leader_election.cc:288] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.644724 3091 ts_tablet_manager.cc:1918] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260927 23:27:59.645198 3091 log.cc:1196] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/517d7bfc225a4372bfd79682a550a0d1
I20260927 23:27:59.645999 3091 ts_tablet_manager.cc:1939] T 517d7bfc225a4372bfd79682a550a0d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
W20260927 23:27:59.649302 2906 leader_election.cc:341] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:59.650609 3092 ts_tablet_manager.cc:1905] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:27:59.652477 3072 tablet_bootstrap.cc:492] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:59.652980 3072 ts_tablet_manager.cc:1397] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.035s user 0.007s sys 0.007s
I20260927 23:27:59.654364 2970 raft_consensus.cc:1261] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Refusing update from remote peer 5ce671f028a34ea4b8f0520c2689c9a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:27:59.656112 3164 raft_consensus.cc:504] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.657313 2971 raft_consensus.cc:2379] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5ce671f028a34ea4b8f0520c2689c9a1 in current term 1: Already voted for candidate de216a4a8e4b4e1b857d8647a3b05f6d in this term.
I20260927 23:27:59.661103 2946 tablet_service.cc:1467] Processing CreateTablet for tablet fac49c5b79e347158621dda52f79e912 (DEFAULT_TABLE table=default.Table_0af4c8ade9d14c7ba8662aa76259b23c [id=efae9c5737164e9d811365b2f6763bb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.642807 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_0cce1d1549d5401c8b6c9966fe2674f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:59.662370 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fac49c5b79e347158621dda52f79e912. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:27:59.663445 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0cce1d1549d5401c8b6c9966fe2674f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:59.648875 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 517d7bfc225a4372bfd79682a550a0d1 (table default.Table_29029b98f7f745888cd86bb69c036043 [id=4a83aea286194e6fac09fde39f326c9f]) successfully deleted
I20260927 23:27:59.668529 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5efbd0211cf3408cb5ea9640d8da6694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.672571 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4929fc9e396e48a189c94586d58aa55f" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:59.673302 2826 raft_consensus.cc:2454] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:59.659795 3072 raft_consensus.cc:348] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.678736 3072 raft_consensus.cc:374] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.679044 3072 raft_consensus.cc:729] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.679659 3072 consensus_queue.cc:260] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:27:59.687081 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 66f2842218fc42e5b1b83314e5f1a158 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66f2842218fc42e5b1b83314e5f1a158 already in progress: creating tablet
W20260927 23:27:59.688037 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 60a2bd7ab1bc4fc29cc4e70d6e6de70d because the tablet was not found. No further retry: Not found: Tablet not found: 60a2bd7ab1bc4fc29cc4e70d6e6de70d
W20260927 23:27:59.689230 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 7096e434ac9f4290ad58ba4f7e7c2513 because the tablet was not found. No further retry: Not found: Tablet not found: 7096e434ac9f4290ad58ba4f7e7c2513
I20260927 23:27:59.681769 3072 ts_tablet_manager.cc:1428] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260927 23:27:59.692014 2386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_1f563f833bd94200a45952661e37e1fe" } new_table_name: "default.Table_0b205c7018ea4694b24bcfaeb93e859a" modify_external_catalogs: true
I20260927 23:27:59.693511 3072 tablet_bootstrap.cc:492] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:59.697225 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39b3bf40c51646d38559f6852ebad6f0" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:59.697929 2970 raft_consensus.cc:2454] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:27:59.699753 3072 tablet_bootstrap.cc:654] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.701951 3196 consensus_queue.cc:1035] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:27:59.708737 2392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_faba7f79481947c1a44139eddeb749a6" } modify_external_catalogs: true
I20260927 23:27:59.709363 2392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_faba7f79481947c1a44139eddeb749a6" } modify_external_catalogs: true
I20260927 23:27:59.710944 2608 leader_election.cc:302] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:27:59.711557 3072 tablet_bootstrap.cc:492] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:27:59.711969 3072 ts_tablet_manager.cc:1397] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.003s
I20260927 23:27:59.711987 3196 raft_consensus.cc:2781] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:59.712363 3196 raft_consensus.cc:482] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:59.712668 3196 raft_consensus.cc:3037] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:59.714628 3072 raft_consensus.cc:348] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.714609 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_9088c5760dc44415bf9a9106a40f5ab4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:59.715301 3072 raft_consensus.cc:374] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.715546 3072 raft_consensus.cc:729] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
W20260927 23:27:59.716670 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9088c5760dc44415bf9a9106a40f5ab4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:59.716325 3072 consensus_queue.cc:260] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.718631 3072 ts_tablet_manager.cc:1428] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260927 23:27:59.719624 3072 tablet_bootstrap.cc:492] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:27:59.725710 3072 tablet_bootstrap.cc:654] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.734448 3196 raft_consensus.cc:504] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.735710 3211 raft_consensus.cc:482] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.736214 3211 raft_consensus.cc:504] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.738225 3211 leader_election.cc:288] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.742559 2385 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 3c80915269094d3e8554c1655a1c98af (table default.Table_52009c39b67d47ea919822fecb9fcc5f [id=d4bab68cc19145baae5fab5691c88c8b]). Expected version 2 got 0
I20260927 23:27:59.743462 2385 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (table default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]). Expected version 1 got 0
W20260927 23:27:59.750067 2907 leader_election.cc:341] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 7096e434ac9f4290ad58ba4f7e7c2513
I20260927 23:27:59.752914 3211 raft_consensus.cc:482] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.754503 3211 raft_consensus.cc:504] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.757737 3211 leader_election.cc:288] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.758781 2946 tablet_service.cc:1467] Processing CreateTablet for tablet a80bff324902498fa5c04fe8cbc05f9d (DEFAULT_TABLE table=default.Table_a5f98162932247a3ae2928ce03a691a4 [id=51372c7a0c1b4c9db9df0e9b0cbbe071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.760114 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a80bff324902498fa5c04fe8cbc05f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.762036 3164 leader_election.cc:288] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.764786 2827 raft_consensus.cc:1261] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Refusing update from remote peer 5ce671f028a34ea4b8f0520c2689c9a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:27:59.766145 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39b3bf40c51646d38559f6852ebad6f0" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:59.768577 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95216fdc2e104a2e9fc4bc424bdbe5e9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:59.772555 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4929fc9e396e48a189c94586d58aa55f" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:59.773154 2828 raft_consensus.cc:3037] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:59.783097 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43096d258a654532906f59d581f5a5d6" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:59.785365 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7096e434ac9f4290ad58ba4f7e7c2513" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:27:59.786765 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95216fdc2e104a2e9fc4bc424bdbe5e9" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:59.756608 2652 tablet_service.cc:1467] Processing CreateTablet for tablet a80bff324902498fa5c04fe8cbc05f9d (DEFAULT_TABLE table=default.Table_a5f98162932247a3ae2928ce03a691a4 [id=51372c7a0c1b4c9db9df0e9b0cbbe071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.800923 3248 mvcc.cc:205] Tried to move back new op lower bound from 7334099679705161728 to 7334099676119166976. Current Snapshot: MvccSnapshot[applied={T|T < 7334099679705161728}]
I20260927 23:27:59.801837 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a80bff324902498fa5c04fe8cbc05f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.801940 3242 raft_consensus.cc:482] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.801755 3253 tablet.cc:1722] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:27:59.802376 3242 raft_consensus.cc:504] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.757752 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43096d258a654532906f59d581f5a5d6" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:59.804591 2674 raft_consensus.cc:2454] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:27:59.762142 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4929fc9e396e48a189c94586d58aa55f" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:27:59.805557 2672 raft_consensus.cc:3037] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:59.813370 3070 tablet_bootstrap.cc:492] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:59.813885 3070 ts_tablet_manager.cc:1397] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.395s user 0.100s sys 0.133s
I20260927 23:27:59.816280 3070 raft_consensus.cc:348] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.817031 3070 raft_consensus.cc:374] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.817272 3070 raft_consensus.cc:729] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.817889 3070 consensus_queue.cc:260] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.819805 3070 ts_tablet_manager.cc:1428] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260927 23:27:59.820731 3070 tablet_bootstrap.cc:492] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:27:59.822597 3242 leader_election.cc:288] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:27:59.822768 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca42b47d8044adc99a82859bf79e606" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:27:59.823438 2674 raft_consensus.cc:2454] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:27:59.822304 2672 raft_consensus.cc:2454] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:27:59.825085 2763 leader_election.cc:302] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:27:59.764249 3112 raft_consensus.cc:504] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:27:59.770092 2609 leader_election.cc:341] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:59.828437 2609 leader_election.cc:302] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:27:59.770619 3163 consensus_queue.cc:1035] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:27:59.830785 3163 raft_consensus.cc:2726] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:59.838557 3167 raft_consensus.cc:348] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.839368 3167 raft_consensus.cc:374] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.839620 3167 raft_consensus.cc:729] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.842175 2673 raft_consensus.cc:2379] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
W20260927 23:27:59.800949 2906 leader_election.cc:341] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:59.859019 3242 raft_consensus.cc:2781] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:59.859440 3242 raft_consensus.cc:482] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:59.860235 3242 raft_consensus.cc:3037] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:59.862183 2907 leader_election.cc:302] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:59.863322 2907 leader_election.cc:302] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:27:59.865204 3211 raft_consensus.cc:2781] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:27:59.865582 3211 raft_consensus.cc:482] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:27:59.865895 3211 raft_consensus.cc:3037] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:59.872510 3164 raft_consensus.cc:2781] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:59.874745 3112 leader_election.cc:288] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:27:59.876220 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 75311us
I20260927 23:27:59.848708 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet a731a36b1da24f559c77bd2b6b484c72 (table default.Table_f4808e8ee6d64bbb8e824ed11976279d [id=94bf26c26f974c258ee084dbc165be72]) successfully deleted
I20260927 23:27:59.882378 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95216fdc2e104a2e9fc4bc424bdbe5e9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:27:59.883410 2970 raft_consensus.cc:2379] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5ce671f028a34ea4b8f0520c2689c9a1 in current term 1: Already voted for candidate de216a4a8e4b4e1b857d8647a3b05f6d in this term.
I20260927 23:27:59.889365 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_1cf287cdda1046558956f74f784c2536"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:27:59.889747 3164 raft_consensus.cc:686] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
W20260927 23:27:59.891242 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1cf287cdda1046558956f74f784c2536 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:27:59.884274 3070 tablet_bootstrap.cc:654] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:27:59.894574 3211 raft_consensus.cc:504] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.898465 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43096d258a654532906f59d581f5a5d6" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:27:59.899121 2673 raft_consensus.cc:3037] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:27:59.887002 3196 leader_election.cc:288] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:27:59.892741 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 122677us
I20260927 23:27:59.909955 2673 raft_consensus.cc:2454] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:27:59.911754 2907 leader_election.cc:302] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:27:59.913136 3257 raft_consensus.cc:2781] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:27:59.914306 3196 raft_consensus.cc:482] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.915258 3196 raft_consensus.cc:504] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.916329 2653 tablet_service.cc:1467] Processing CreateTablet for tablet e104d49543934e5db3b188e1db065cdc (DEFAULT_TABLE table=default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.917593 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e104d49543934e5db3b188e1db065cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:27:59.926164 3070 tablet_bootstrap.cc:492] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:27:59.926664 3070 ts_tablet_manager.cc:1397] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.106s user 0.014s sys 0.000s
I20260927 23:27:59.929086 3070 raft_consensus.cc:348] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.929764 3070 raft_consensus.cc:374] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:27:59.930022 3070 raft_consensus.cc:729] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:27:59.930593 2946 tablet_service.cc:1467] Processing CreateTablet for tablet e104d49543934e5db3b188e1db065cdc (DEFAULT_TABLE table=default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:27:59.931768 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e104d49543934e5db3b188e1db065cdc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:27:59.882361 2906 leader_election.cc:341] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:59.932144 2906 leader_election.cc:302] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:59.937412 3259 raft_consensus.cc:2726] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:27:59.937690 2379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.62:38773, user_credentials={real_user=slave}} blocked reactor thread for 60300.7us
I20260927 23:27:59.890806 3164 consensus_queue.cc:237] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.949226 3257 raft_consensus.cc:686] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:27:59.950307 3257 consensus_queue.cc:237] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:27:59.957610 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 75288us
W20260927 23:27:59.958475 2906 leader_election.cc:341] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:59.958976 2906 leader_election.cc:302] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:27:59.960098 2906 leader_election.cc:341] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:27:59.961408 2906 leader_election.cc:302] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:27:59.966456 2828 raft_consensus.cc:2454] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:27:59.964875 3211 leader_election.cc:288] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:27:59.965512 3260 raft_consensus.cc:2726] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260927 23:27:59.983785 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43096d258a654532906f59d581f5a5d6" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:27:59.994540 3259 raft_consensus.cc:2726] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:27:59.840258 3167 consensus_queue.cc:260] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:27:59.995139 3259 raft_consensus.cc:482] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:27:59.998078 3167 ts_tablet_manager.cc:1428] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.774s user 0.008s sys 0.000s
I20260927 23:27:59.995586 3259 raft_consensus.cc:504] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.002173 3259 leader_election.cc:288] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:00.003589 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39b3bf40c51646d38559f6852ebad6f0" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:00.004449 2970 raft_consensus.cc:3037] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:00.013481 2380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 9cce47b050b445dfba7be5ea3b6a8636 (table default.Table_746dd85f958f4346bb7c3484f59e75f4 [id=b12aae65f933476ab57974d4b328a266], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9cce47b050b445dfba7be5ea3b6a8636 consensus state has no leader
I20260927 23:28:00.027983 2970 raft_consensus.cc:2454] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:00.035066 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45dd55b5a9d94ff0b84a799174ab5359" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:27:59.935266 3070 consensus_queue.cc:260] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.038161 2970 raft_consensus.cc:2454] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
W20260927 23:28:00.045619 2906 leader_election.cc:341] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:00.055588 3070 ts_tablet_manager.cc:1428] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.129s user 0.008s sys 0.000s
I20260927 23:28:00.038160 3196 leader_election.cc:288] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:00.057642 3252 tablet.cc:1722] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:00.074524 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fc2f03baae4f70b38b8079deb8a854" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:00.075362 2673 raft_consensus.cc:2379] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate abe182b2afc64d908f692ef45eae1d51 in this term.
I20260927 23:28:00.084574 2608 leader_election.cc:302] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:00.085618 2608 leader_election.cc:302] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:00.091485 2609 leader_election.cc:302] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:00.098112 3070 tablet_bootstrap.cc:492] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:00.102095 3250 raft_consensus.cc:2781] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:00.102634 3250 raft_consensus.cc:686] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:00.103442 3250 consensus_queue.cc:237] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.104527 3251 raft_consensus.cc:2781] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:00.104842 3251 raft_consensus.cc:482] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:00.105134 3251 raft_consensus.cc:3037] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:00.116955 3112 raft_consensus.cc:2741] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260927 23:28:00.123139 3252 tablet.cc:1722] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260927 23:28:00.125326 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 47095.7us
W20260927 23:28:00.127797 2609 leader_election.cc:341] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:00.128759 2609 leader_election.cc:341] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:00.142745 3251 raft_consensus.cc:504] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.156697 3253 tablet.cc:1722] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:00.165153 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 2d837daee7a04b338813519acf53dfae (DEFAULT_TABLE table=default.Table_5f156a98d2694e9aa22d5bcce0e0890d [id=5e610846ba4f4cd7907236b723747067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:00.166666 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d837daee7a04b338813519acf53dfae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:00.176736 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 933d4c631a1345f9a117b87d0e669117 (DEFAULT_TABLE table=default.Table_789dc9b24bea430295966df216636877 [id=a002f07797a040eeadf50f46ae5533fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:00.183058 3112 consensus_queue.cc:1035] T f5cec41645b24e7d99e3dae3f2e8a22f P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:00.184417 3196 consensus_queue.cc:1035] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:00.189558 3072 tablet_bootstrap.cc:492] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:00.190016 3072 ts_tablet_manager.cc:1397] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.471s user 0.104s sys 0.194s
I20260927 23:28:00.185439 3256 raft_consensus.cc:482] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:00.191494 3256 raft_consensus.cc:504] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.193953 3256 leader_election.cc:288] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:00.197934 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 933d4c631a1345f9a117b87d0e669117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:00.009816 3255 consensus_queue.cc:1035] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260927 23:28:00.201385 3167 ts_tablet_manager.cc:1460] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51: Tablet startup took 977ms
I20260927 23:28:00.017102 2803 tablet_service.cc:1467] Processing CreateTablet for tablet fac49c5b79e347158621dda52f79e912 (DEFAULT_TABLE table=default.Table_0af4c8ade9d14c7ba8662aa76259b23c [id=efae9c5737164e9d811365b2f6763bb2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:00.019178 2807 tablet_service.cc:1467] Processing CreateTablet for tablet a80bff324902498fa5c04fe8cbc05f9d (DEFAULT_TABLE table=default.Table_a5f98162932247a3ae2928ce03a691a4 [id=51372c7a0c1b4c9db9df0e9b0cbbe071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:00.021106 2802 tablet_service.cc:1467] Processing CreateTablet for tablet e104d49543934e5db3b188e1db065cdc (DEFAULT_TABLE table=default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:00.201731 3167 ts_tablet_manager.cc:1462] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51: Trace:
0927 23:27:57.671632 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:57.680395 (+ 8763us) service_pool.cc:227] Handling call
0927 23:27:57.681861 (+ 1466us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:27:57.681950 (+ 89us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:27:57.710599 (+ 28649us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:57.766441 (+ 55842us) tablet_replica.cc:205] Creating consensus instance
0927 23:27:57.766806 (+ 365us) inbound_call.cc:173] Queueing success response
0927 23:27:58.885398 (+1118592us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0927 23:27:59.223630 (+338232us) ts_tablet_manager.cc:1429] Starting tablet replica
0927 23:27:59.324992 (+101362us) tablet_replica.cc:259] Starting instrumentation
0927 23:27:59.834762 (+509770us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":15307,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":186,"tablet-open.queue_time_us":1118663}
I20260927 23:28:00.203158 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e104d49543934e5db3b188e1db065cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:00.203969 3167 tablet_bootstrap.cc:492] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:00.089938 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74fc2f03baae4f70b38b8079deb8a854" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:00.215085 2906 leader_election.cc:302] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:00.216383 3257 raft_consensus.cc:3037] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:00.222057 3090 ts_tablet_manager.cc:1918] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:00.222481 3090 log.cc:1196] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/a731a36b1da24f559c77bd2b6b484c72
I20260927 23:28:00.223131 3090 ts_tablet_manager.cc:1939] T a731a36b1da24f559c77bd2b6b484c72 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:00.236784 3072 raft_consensus.cc:348] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.237609 3072 raft_consensus.cc:374] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:00.237869 3072 raft_consensus.cc:729] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:00.238575 3072 consensus_queue.cc:260] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.242208 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a80bff324902498fa5c04fe8cbc05f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:00.257838 3090 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 47) took 1542 ms. Trace:
I20260927 23:28:00.258440 3090 rpcz_store.cc:276] 0927 23:27:58.715605 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:58.717047 (+ 1442us) service_pool.cc:227] Handling call
0927 23:27:59.100258 (+383211us) ts_tablet_manager.cc:1145] Deleting tablet a731a36b1da24f559c77bd2b6b484c72
0927 23:27:59.100339 (+ 81us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:27:59.747792 (+647453us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:00.220601 (+472809us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:00.257744 (+ 37143us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":382711}
I20260927 23:28:00.261180 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fac49c5b79e347158621dda52f79e912. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:00.263424 3092 ts_tablet_manager.cc:1918] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260927 23:28:00.268627 3092 log.cc:1196] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/517d7bfc225a4372bfd79682a550a0d1
I20260927 23:28:00.271165 3092 ts_tablet_manager.cc:1939] T 517d7bfc225a4372bfd79682a550a0d1 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:00.260499 3164 raft_consensus.cc:482] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:00.276435 3164 raft_consensus.cc:504] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.278683 3164 leader_election.cc:288] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:00.262315 3072 ts_tablet_manager.cc:1428] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.072s user 0.004s sys 0.004s
I20260927 23:28:00.280033 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbfa55a66b8042ed8592fbff31e92d55" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:00.280547 3072 tablet_bootstrap.cc:492] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
W20260927 23:28:00.282506 2906 leader_election.cc:341] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:00.282948 3090 tablet_replica.cc:330] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:00.283643 3167 tablet_bootstrap.cc:654] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:00.284824 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 121479us
I20260927 23:28:00.286329 3090 raft_consensus.cc:2229] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:00.296636 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet a731a36b1da24f559c77bd2b6b484c72 (table default.Table_f4808e8ee6d64bbb8e824ed11976279d [id=94bf26c26f974c258ee084dbc165be72]) successfully deleted
I20260927 23:28:00.286762 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828f870365244777b59e8a0756d7d4c9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:00.300632 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca42b47d8044adc99a82859bf79e606" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:00.300573 2945 tablet_service.cc:1467] Processing CreateTablet for tablet 933d4c631a1345f9a117b87d0e669117 (DEFAULT_TABLE table=default.Table_789dc9b24bea430295966df216636877 [id=a002f07797a040eeadf50f46ae5533fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:00.300737 2972 raft_consensus.cc:2454] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:00.301343 2970 raft_consensus.cc:2454] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:28:00.302039 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 933d4c631a1345f9a117b87d0e669117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:00.303062 3090 raft_consensus.cc:2258] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:00.304962 3090 ts_tablet_manager.cc:1905] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:00.293679 3257 raft_consensus.cc:2726] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:00.305277 2947 tablet_service.cc:1467] Processing CreateTablet for tablet 84d37dba66b84d05abf675d3eaf39a30 (DEFAULT_TABLE table=default.Table_67d5684dd07845a3a70af79c67fed142 [id=6671aa9b745f4863982b1d2dc75e5b63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:00.306607 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84d37dba66b84d05abf675d3eaf39a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:00.297883 3072 tablet_bootstrap.cc:654] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:00.298799 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 2d837daee7a04b338813519acf53dfae (DEFAULT_TABLE table=default.Table_5f156a98d2694e9aa22d5bcce0e0890d [id=5e610846ba4f4cd7907236b723747067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:00.313539 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d837daee7a04b338813519acf53dfae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:00.333608 3070 tablet_bootstrap.cc:654] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:00.342074 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_e57880e3ffa64d3b876813b8c2246002"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:00.344370 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e57880e3ffa64d3b876813b8c2246002 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:00.354604 3072 tablet_bootstrap.cc:492] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:00.355168 3072 ts_tablet_manager.cc:1397] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.075s user 0.009s sys 0.001s
I20260927 23:28:00.366021 3072 raft_consensus.cc:348] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.366882 3072 raft_consensus.cc:374] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:00.367151 3072 raft_consensus.cc:729] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:00.367830 3072 consensus_queue.cc:260] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.370133 3072 ts_tablet_manager.cc:1428] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.015s user 0.004s sys 0.004s
I20260927 23:28:00.379990 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_2ec8e4445f1d425ea0a7dae9f9d6200b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:00.382299 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2ec8e4445f1d425ea0a7dae9f9d6200b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:00.383947 3090 ts_tablet_manager.cc:1918] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260927 23:28:00.389634 3090 log.cc:1196] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/517d7bfc225a4372bfd79682a550a0d1
I20260927 23:28:00.390764 3090 ts_tablet_manager.cc:1939] T 517d7bfc225a4372bfd79682a550a0d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:00.429801 3072 tablet_bootstrap.cc:492] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:00.431190 3090 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 51) took 1165 ms. Trace:
I20260927 23:28:00.431569 3090 rpcz_store.cc:276] 0927 23:27:59.265885 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:59.266023 (+ 138us) service_pool.cc:227] Handling call
0927 23:28:00.282672 (+1016649us) ts_tablet_manager.cc:1145] Deleting tablet 517d7bfc225a4372bfd79682a550a0d1
0927 23:28:00.282719 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:00.331801 (+ 49082us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:00.368656 (+ 36855us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:00.431117 (+ 62461us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1016011}
I20260927 23:28:00.435966 3072 tablet_bootstrap.cc:654] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:00.452008 3072 tablet_bootstrap.cc:492] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:00.452566 3072 ts_tablet_manager.cc:1397] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.005s
I20260927 23:28:00.455150 3072 raft_consensus.cc:348] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.455945 3072 raft_consensus.cc:374] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:00.456269 3072 raft_consensus.cc:729] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:00.456962 3072 consensus_queue.cc:260] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.469955 3072 ts_tablet_manager.cc:1428] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260927 23:28:00.487100 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95216fdc2e104a2e9fc4bc424bdbe5e9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:00.489230 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39b3bf40c51646d38559f6852ebad6f0" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:00.494093 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbfa55a66b8042ed8592fbff31e92d55" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:00.499074 2673 raft_consensus.cc:2454] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:00.502694 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45dd55b5a9d94ff0b84a799174ab5359" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:00.503424 2824 raft_consensus.cc:2454] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
W20260927 23:28:00.410048 2379 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.62:38773, user_credentials={real_user=slave}} blocked reactor thread for 363613us
I20260927 23:28:00.508430 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 84d37dba66b84d05abf675d3eaf39a30 (DEFAULT_TABLE table=default.Table_67d5684dd07845a3a70af79c67fed142 [id=6671aa9b745f4863982b1d2dc75e5b63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:00.509801 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84d37dba66b84d05abf675d3eaf39a30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:00.512693 3251 leader_election.cc:288] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:28:00.523646 2378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111) [suppressed 8 similar messages]
I20260927 23:28:00.460057 3256 raft_consensus.cc:482] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:00.564339 3270 raft_consensus.cc:482] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:28:00.565070 3270 raft_consensus.cc:504] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:28:00.558043 3256 raft_consensus.cc:504] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.570819 3257 raft_consensus.cc:482] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:00.571658 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 517d7bfc225a4372bfd79682a550a0d1 (table default.Table_29029b98f7f745888cd86bb69c036043 [id=4a83aea286194e6fac09fde39f326c9f]) successfully deleted
I20260927 23:28:00.571429 3257 raft_consensus.cc:504] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.572115 3256 leader_election.cc:288] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:00.573400 3257 leader_election.cc:288] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:28:00.577970 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 84586.2us
I20260927 23:28:00.578748 2608 leader_election.cc:302] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:00.580710 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45dd55b5a9d94ff0b84a799174ab5359" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:00.581372 2970 raft_consensus.cc:3037] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:00.585036 2907 leader_election.cc:302] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:00.589216 3257 raft_consensus.cc:2781] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:00.589602 3257 raft_consensus.cc:482] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:00.589908 3257 raft_consensus.cc:3037] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:00.595291 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdfe7faa1a4d5d8e340f09f1cae93e" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:00.596024 2971 raft_consensus.cc:2454] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:00.599231 3090 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 53) took 1137 ms. Trace:
I20260927 23:28:00.599769 3072 tablet_bootstrap.cc:492] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:00.599606 3090 rpcz_store.cc:276] 0927 23:27:59.462099 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:59.462278 (+ 179us) service_pool.cc:227] Handling call
0927 23:28:00.432701 (+970423us) ts_tablet_manager.cc:1145] Deleting tablet 66f2842218fc42e5b1b83314e5f1a158
0927 23:28:00.598834 (+166133us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:00.599152 (+ 318us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":968638}
I20260927 23:28:00.600999 3251 raft_consensus.cc:2781] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:00.601459 3251 raft_consensus.cc:482] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:00.602424 3251 raft_consensus.cc:3037] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:00.607115 3072 tablet_bootstrap.cc:654] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:00.615283 2970 raft_consensus.cc:2454] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:00.635896 2608 leader_election.cc:302] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:00.636958 3250 raft_consensus.cc:2781] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:00.637403 3250 raft_consensus.cc:482] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:00.637784 3250 raft_consensus.cc:3037] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:00.646349 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.60:41571
--webserver_interface=127.2.57.60
--webserver_port=39731
--builtin_ntp_servers=127.2.57.20:37745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--hive_metastore_notification_log_poll_period_seconds=1
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260927 23:28:00.649832 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdfe7faa1a4d5d8e340f09f1cae93e" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:00.670496 3251 raft_consensus.cc:504] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.621907 3072 tablet_bootstrap.cc:492] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:00.672617 3072 ts_tablet_manager.cc:1397] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.073s user 0.004s sys 0.010s
I20260927 23:28:00.675195 3072 raft_consensus.cc:348] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.675952 3072 raft_consensus.cc:374] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:00.676206 3072 raft_consensus.cc:729] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:00.677323 3072 consensus_queue.cc:260] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.679438 3072 ts_tablet_manager.cc:1428] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.007s user 0.003s sys 0.001s
I20260927 23:28:00.680477 3072 tablet_bootstrap.cc:492] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:00.682664 3257 raft_consensus.cc:504] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.605589 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdfe7faa1a4d5d8e340f09f1cae93e" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:00.611830 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 2d837daee7a04b338813519acf53dfae (DEFAULT_TABLE table=default.Table_5f156a98d2694e9aa22d5bcce0e0890d [id=5e610846ba4f4cd7907236b723747067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:00.697350 3072 tablet_bootstrap.cc:654] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:00.699247 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 4 candidate_status { last_received { term: 3 index: 122 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:28:00.716517 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 517d7bfc225a4372bfd79682a550a0d1 (table default.Table_29029b98f7f745888cd86bb69c036043 [id=4a83aea286194e6fac09fde39f326c9f]) successfully deleted
I20260927 23:28:00.718468 3072 tablet_bootstrap.cc:492] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:00.718927 3072 ts_tablet_manager.cc:1397] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.039s user 0.012s sys 0.001s
I20260927 23:28:00.721109 3257 leader_election.cc:288] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:00.721616 3072 raft_consensus.cc:348] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:28:00.722038 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 66f2842218fc42e5b1b83314e5f1a158 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66f2842218fc42e5b1b83314e5f1a158 already in progress: creating tablet
I20260927 23:28:00.722325 3072 raft_consensus.cc:374] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:00.722592 3072 raft_consensus.cc:729] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:00.723225 3072 consensus_queue.cc:260] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.673611 2608 leader_election.cc:302] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:00.725310 3072 ts_tablet_manager.cc:1428] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
W20260927 23:28:00.725667 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 52175.1us
I20260927 23:28:00.726262 3072 tablet_bootstrap.cc:492] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:00.727205 3273 raft_consensus.cc:2781] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:00.728564 3273 raft_consensus.cc:686] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:00.729453 3273 consensus_queue.cc:237] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.731640 3072 tablet_bootstrap.cc:654] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:00.722720 3250 raft_consensus.cc:504] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:28:00.692063 3242 consensus_meta.cc:287] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51: Time spent flushing consensus metadata: real 0.831s user 0.009s sys 0.000s
I20260927 23:28:00.632404 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828f870365244777b59e8a0756d7d4c9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:00.635283 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 84d37dba66b84d05abf675d3eaf39a30 (DEFAULT_TABLE table=default.Table_67d5684dd07845a3a70af79c67fed142 [id=6671aa9b745f4863982b1d2dc75e5b63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:00.638099 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "45dd55b5a9d94ff0b84a799174ab5359" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:00.739470 2823 raft_consensus.cc:3037] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:00.740643 2673 raft_consensus.cc:2379] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:00.748229 2823 raft_consensus.cc:2454] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:00.664209 3167 tablet_bootstrap.cc:492] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:00.680749 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdfe7faa1a4d5d8e340f09f1cae93e" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:00.751410 3167 ts_tablet_manager.cc:1397] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.548s user 0.015s sys 0.008s
I20260927 23:28:00.755756 3242 raft_consensus.cc:504] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.761343 3072 tablet_bootstrap.cc:492] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:00.761859 3072 ts_tablet_manager.cc:1397] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.036s user 0.013s sys 0.001s
I20260927 23:28:00.762420 3070 tablet_bootstrap.cc:492] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:00.762986 3070 ts_tablet_manager.cc:1397] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.665s user 0.015s sys 0.004s
I20260927 23:28:00.764506 3072 raft_consensus.cc:348] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.765278 3072 raft_consensus.cc:374] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:00.765584 3072 raft_consensus.cc:729] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:00.766274 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbfa55a66b8042ed8592fbff31e92d55" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:00.766479 3072 consensus_queue.cc:260] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:28:00.768714 2906 leader_election.cc:341] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:00.771060 3250 leader_election.cc:288] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:00.691215 2806 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42492 (request call id 48) took 1474 ms. Trace:
I20260927 23:28:00.771921 2806 rpcz_store.cc:276] 0927 23:27:59.217136 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:59.220425 (+ 3289us) service_pool.cc:227] Handling call
0927 23:27:59.565244 (+344819us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b5800001898. stack: 00007fbefd112033 00007fbefd111f30 00007fbefcc93534 00000000004db2b4 00000000004db249 00007fbefcf88f11 00007fbefd10cb7c 00007fbefd10c73b 00007fbf05fa3b09 00007fbf05000730 00007fbf050e2d21 00007fbf050c01e3 00007fbf05071869 00007fbf03763c44 00007fbf03763bd2 00007fbf03763b01
0927 23:27:59.582253 (+ 17009us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:27:59.587721 (+ 5468us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:27:59.974297 (+386576us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:00.541487 (+567190us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:00.691129 (+149642us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":727693184}
I20260927 23:28:00.773654 3072 ts_tablet_manager.cc:1428] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
W20260927 23:28:00.542224 2763 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.716s user 0.001s sys 0.014s
I20260927 23:28:00.692600 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d837daee7a04b338813519acf53dfae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:00.774641 3072 tablet_bootstrap.cc:492] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:00.630544 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 933d4c631a1345f9a117b87d0e669117 (DEFAULT_TABLE table=default.Table_789dc9b24bea430295966df216636877 [id=a002f07797a040eeadf50f46ae5533fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:00.693102 2828 raft_consensus.cc:2454] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:00.781229 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 933d4c631a1345f9a117b87d0e669117. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:00.786402 2906 leader_election.cc:302] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:00.787456 3257 raft_consensus.cc:2781] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:00.787961 3257 raft_consensus.cc:482] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:00.788339 3257 raft_consensus.cc:3037] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:00.792246 3072 tablet_bootstrap.cc:654] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:00.800724 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 84d37dba66b84d05abf675d3eaf39a30. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:00.804586 2763 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.978s user 0.004s sys 0.015s
I20260927 23:28:00.805233 3257 raft_consensus.cc:504] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:28:00.808023 2763 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 981799us
I20260927 23:28:00.808845 3257 leader_election.cc:288] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:00.809600 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdfe7faa1a4d5d8e340f09f1cae93e" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:00.810220 2828 raft_consensus.cc:3037] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:00.813033 3072 tablet_bootstrap.cc:492] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:00.813516 3072 ts_tablet_manager.cc:1397] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.039s user 0.010s sys 0.002s
I20260927 23:28:00.816057 3072 raft_consensus.cc:348] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.816992 3072 raft_consensus.cc:374] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:00.817258 3072 raft_consensus.cc:729] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:00.817924 3072 consensus_queue.cc:260] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.819105 2828 raft_consensus.cc:2454] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:00.820487 2906 leader_election.cc:302] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:00.832940 3164 raft_consensus.cc:2781] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:00.833498 3164 raft_consensus.cc:686] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:00.834230 3072 ts_tablet_manager.cc:1428] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.020s user 0.007s sys 0.000s
I20260927 23:28:00.834369 3164 consensus_queue.cc:237] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.835258 3072 tablet_bootstrap.cc:492] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:00.848956 3072 tablet_bootstrap.cc:654] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:00.734040 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 66f2842218fc42e5b1b83314e5f1a158 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 66f2842218fc42e5b1b83314e5f1a158 already in progress: creating tablet
W20260927 23:28:00.851300 2378 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 117310us
I20260927 23:28:00.855567 3257 consensus_queue.cc:1035] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260927 23:28:00.852119 2379 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 130136us
I20260927 23:28:00.877172 2393 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 98) took 1123 ms. Trace:
I20260927 23:28:00.877548 2393 rpcz_store.cc:276] 0927 23:27:59.754125 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:59.888828 (+134703us) service_pool.cc:227] Handling call
0927 23:27:59.891701 (+ 2873us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:27:59.898550 (+ 6849us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:00.876451 (+977901us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:00.876794 (+ 343us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:00.877081 (+ 287us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:00.033837 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:00.034596 (+ 759us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:00.034627 (+ 31us) write_op.cc:435] Acquired schema lock
0927 23:28:00.034642 (+ 15us) tablet.cc:660] Decoding operations
0927 23:28:00.034980 (+ 338us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:00.035022 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:00.035046 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:00.035413 (+ 367us) tablet.cc:699] Row locks acquired
0927 23:28:00.035429 (+ 16us) write_op.cc:260] PREPARE: finished
0927 23:28:00.035594 (+ 165us) write_op.cc:270] Start()
0927 23:28:00.035698 (+ 104us) write_op.cc:276] Timestamp: P: 1790551680035517 usec, L: 0
0927 23:28:00.035716 (+ 18us) op_driver.cc:331] REPLICATION: starting
0927 23:28:00.036572 (+ 856us) log.cc:842] Serialized 432 byte log entry
0927 23:28:00.873031 (+836459us) op_driver.cc:444] REPLICATION: finished
0927 23:28:00.873889 (+ 858us) write_op.cc:301] APPLY: starting
0927 23:28:00.873961 (+ 72us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:00.874075 (+ 114us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:00.875038 (+ 963us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:00.875458 (+ 420us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:00.875601 (+ 143us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:00.875638 (+ 37us) write_op.cc:312] APPLY: finished
0927 23:28:00.875861 (+ 223us) log.cc:842] Serialized 28 byte log entry
0927 23:28:00.876037 (+ 176us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:00.876111 (+ 74us) write_op.cc:454] Released schema lock
0927 23:28:00.876144 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:00.876350 (+ 206us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:00.876373 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":673,"apply.run_cpu_time_us":2660,"apply.run_wall_time_us":2660,"num_ops":2,"prepare.queue_time_us":133784,"prepare.run_cpu_time_us":2499,"prepare.run_wall_time_us":350505,"replication_time_us":837220}]]}
I20260927 23:28:00.888101 3257 raft_consensus.cc:482] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:00.888629 3257 raft_consensus.cc:504] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.892129 3257 leader_election.cc:288] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:00.897193 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66f2842218fc42e5b1b83314e5f1a158" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:00.913600 3072 tablet_bootstrap.cc:492] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:00.914173 3072 ts_tablet_manager.cc:1397] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.079s user 0.010s sys 0.001s
I20260927 23:28:00.915688 3270 leader_election.cc:288] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:28:00.917707 3072 raft_consensus.cc:348] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.918434 3072 raft_consensus.cc:374] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:00.918684 3072 raft_consensus.cc:729] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:00.919286 3072 consensus_queue.cc:260] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.921835 3072 ts_tablet_manager.cc:1428] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260927 23:28:00.922827 3072 tablet_bootstrap.cc:492] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
W20260927 23:28:00.924926 2906 leader_election.cc:341] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:00.928318 2303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111) [suppressed 13 similar messages]
I20260927 23:28:00.937436 3257 raft_consensus.cc:482] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:00.937997 3257 raft_consensus.cc:504] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.941141 3257 leader_election.cc:288] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:00.942449 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d299ca52cfc493499b5eaf0acdeb5f9" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:00.944196 2906 leader_election.cc:341] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:00.950219 3072 tablet_bootstrap.cc:654] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:00.952545 2906 consensus_peers.cc:487] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:00.757870 3251 leader_election.cc:288] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:28:00.974525 2609 leader_election.cc:341] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:00.975036 2609 leader_election.cc:302] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:00.975824 3250 raft_consensus.cc:2726] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260927 23:28:00.976169 2609 leader_election.cc:341] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:00.985549 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fbfa55a66b8042ed8592fbff31e92d55" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:00.986133 2672 raft_consensus.cc:3037] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:00.985265 3070 raft_consensus.cc:348] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:28:00.985308 2303 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.60:41571, user_credentials={real_user=slave}} blocked reactor thread for 57061.2us
I20260927 23:28:00.989009 3070 raft_consensus.cc:374] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:00.989291 3070 raft_consensus.cc:729] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:00.990016 3070 consensus_queue.cc:260] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:00.993332 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdfe7faa1a4d5d8e340f09f1cae93e" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:00.993757 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66f2842218fc42e5b1b83314e5f1a158" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:01.001688 2674 raft_consensus.cc:2379] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:01.003090 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828f870365244777b59e8a0756d7d4c9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:01.003803 2972 raft_consensus.cc:3037] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:01.012513 2972 raft_consensus.cc:2454] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
W20260927 23:28:01.053190 2380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 9cce47b050b445dfba7be5ea3b6a8636 (table default.Table_746dd85f958f4346bb7c3484f59e75f4 [id=b12aae65f933476ab57974d4b328a266], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9cce47b050b445dfba7be5ea3b6a8636 consensus state has no leader
W20260927 23:28:01.053010 2303 leader_election.cc:334] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571): Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111)
I20260927 23:28:01.053582 2303 leader_election.cc:302] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bee21c28e51643189178a3c851672546; no voters: 0cdd64d48a684d5ebccc2e2e8a4e3150, 19a88ff0709d4266b933cc526577127f
I20260927 23:28:01.054596 3270 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:28:01.052194 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d299ca52cfc493499b5eaf0acdeb5f9" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:01.062037 3070 ts_tablet_manager.cc:1428] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.299s user 0.004s sys 0.003s
I20260927 23:28:01.065400 3070 tablet_bootstrap.cc:492] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:01.069231 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdfe7faa1a4d5d8e340f09f1cae93e" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
W20260927 23:28:01.076989 2907 leader_election.cc:341] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.077298 2608 leader_election.cc:302] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:01.077508 2907 leader_election.cc:302] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:01.078714 2907 leader_election.cc:341] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.079177 2907 leader_election.cc:302] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:01.079993 3257 raft_consensus.cc:2726] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:01.080554 3257 raft_consensus.cc:2726] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:01.082962 3112 raft_consensus.cc:2781] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:01.083465 3112 raft_consensus.cc:686] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:01.084333 3112 consensus_queue.cc:237] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.095877 2672 raft_consensus.cc:2454] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:01.106386 3164 consensus_queue.cc:1035] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:01.107647 2907 leader_election.cc:302] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:01.112319 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_047c5f89196f446e86279aae36cf7f99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:01.114084 3070 tablet_bootstrap.cc:654] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:01.114256 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_047c5f89196f446e86279aae36cf7f99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:01.117800 3164 raft_consensus.cc:2781] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:01.120360 3164 raft_consensus.cc:686] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:01.122140 3164 consensus_queue.cc:237] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.122141 3302 tablet.cc:1722] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:01.130757 3301 tablet.cc:1722] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:01.140516 3072 tablet_bootstrap.cc:492] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:01.141032 3072 ts_tablet_manager.cc:1397] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.218s user 0.011s sys 0.013s
I20260927 23:28:01.143752 3072 raft_consensus.cc:348] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.157253 3072 raft_consensus.cc:374] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:01.157572 3072 raft_consensus.cc:729] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:01.167112 3072 consensus_queue.cc:260] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.173024 3072 ts_tablet_manager.cc:1428] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.032s user 0.004s sys 0.002s
I20260927 23:28:01.173987 3072 tablet_bootstrap.cc:492] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:01.196159 2391 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 101) took 1014 ms. Trace:
I20260927 23:28:01.199450 3070 tablet_bootstrap.cc:492] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:01.197209 2391 rpcz_store.cc:276] 0927 23:28:00.182285 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:01.111649 (+929364us) service_pool.cc:227] Handling call
0927 23:28:01.114710 (+ 3061us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:01.126444 (+ 11734us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:01.195414 (+ 68970us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:01.195797 (+ 383us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:01.196086 (+ 289us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:01.137813 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:01.138258 (+ 445us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:01.138309 (+ 51us) write_op.cc:435] Acquired schema lock
0927 23:28:01.138325 (+ 16us) tablet.cc:660] Decoding operations
0927 23:28:01.138650 (+ 325us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:01.138703 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:01.138728 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:01.138948 (+ 220us) tablet.cc:699] Row locks acquired
0927 23:28:01.138963 (+ 15us) write_op.cc:260] PREPARE: finished
0927 23:28:01.139109 (+ 146us) write_op.cc:270] Start()
0927 23:28:01.139207 (+ 98us) write_op.cc:276] Timestamp: P: 1790551681139036 usec, L: 0
0927 23:28:01.139225 (+ 18us) op_driver.cc:331] REPLICATION: starting
0927 23:28:01.139881 (+ 656us) log.cc:842] Serialized 432 byte log entry
0927 23:28:01.192622 (+ 52741us) op_driver.cc:444] REPLICATION: finished
0927 23:28:01.192898 (+ 276us) write_op.cc:301] APPLY: starting
0927 23:28:01.192968 (+ 70us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:01.193081 (+ 113us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:01.193184 (+ 103us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:01.193611 (+ 427us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:01.193753 (+ 142us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:01.193793 (+ 40us) write_op.cc:312] APPLY: finished
0927 23:28:01.194014 (+ 221us) log.cc:842] Serialized 28 byte log entry
0927 23:28:01.194176 (+ 162us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:01.194255 (+ 79us) write_op.cc:454] Released schema lock
0927 23:28:01.194297 (+ 42us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:01.194492 (+ 195us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:01.194517 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":85,"apply.run_cpu_time_us":1812,"apply.run_wall_time_us":1812,"num_ops":2,"prepare.queue_time_us":9819,"prepare.run_cpu_time_us":2344,"prepare.run_wall_time_us":2344,"replication_time_us":53311}]]}
I20260927 23:28:01.199906 3070 ts_tablet_manager.cc:1397] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.135s user 0.005s sys 0.005s
I20260927 23:28:01.201211 2391 master_service.cc:946] ReplaceTablet: received request to replace tablet 3c80915269094d3e8554c1655a1c98af from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:01.202376 3070 raft_consensus.cc:348] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.203107 3070 raft_consensus.cc:374] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:01.203353 3070 raft_consensus.cc:729] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:01.207916 3251 consensus_queue.cc:1035] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:01.211090 3164 consensus_queue.cc:1035] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:28:01.223881 2378 consensus_peers.cc:487] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f -> Peer 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571): Couldn't send request to peer 0cdd64d48a684d5ebccc2e2e8a4e3150. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260927 23:28:01.203976 3070 consensus_queue.cc:260] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.235139 3251 raft_consensus.cc:482] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:01.235757 3251 raft_consensus.cc:504] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.240427 3251 leader_election.cc:288] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:01.242169 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66f2842218fc42e5b1b83314e5f1a158" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:01.242894 2972 raft_consensus.cc:2454] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:01.246757 2608 leader_election.cc:302] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:01.251212 3112 raft_consensus.cc:2781] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:01.251616 3112 raft_consensus.cc:482] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:01.251916 3112 raft_consensus.cc:3037] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:01.260017 2609 leader_election.cc:341] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.261216 3112 raft_consensus.cc:504] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.264506 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca42b47d8044adc99a82859bf79e606" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:01.265250 2972 raft_consensus.cc:3037] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:01.274159 2972 raft_consensus.cc:2454] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:28:01.289327 3112 leader_election.cc:288] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:01.277926 3070 ts_tablet_manager.cc:1428] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.078s user 0.004s sys 0.004s
I20260927 23:28:01.291998 3070 tablet_bootstrap.cc:492] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:01.293577 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66f2842218fc42e5b1b83314e5f1a158" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:01.295734 2971 raft_consensus.cc:3037] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:01.279788 3301 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:54604 (request call id 70) took 1010 ms. Trace:
I20260927 23:28:01.304437 3301 rpcz_store.cc:276] 0927 23:28:00.269117 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:00.269286 (+ 169us) service_pool.cc:227] Handling call
0927 23:28:01.279706 (+1010420us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:00.270524 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0927 23:28:00.270917 (+ 393us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0927 23:28:00.270947 (+ 30us) alter_schema_op.cc:65] Acquired schema lock
0927 23:28:00.271007 (+ 60us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0927 23:28:00.271146 (+ 139us) alter_schema_op.cc:126] START. Timestamp: 1790551680271044
0927 23:28:00.271169 (+ 23us) op_driver.cc:331] REPLICATION: starting
0927 23:28:00.271781 (+ 612us) log.cc:842] Serialized 173 byte log entry
0927 23:28:01.002481 (+730700us) op_driver.cc:444] REPLICATION: finished
0927 23:28:01.130488 (+128007us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0927 23:28:01.133849 (+ 3361us) log.cc:842] Serialized 5 byte log entry
0927 23:28:01.232412 (+ 98563us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:01.278855 (+ 46443us) log.cc:842] Serialized 14 byte log entry
0927 23:28:01.279549 (+ 694us) alter_schema_op.cc:71] Released schema lock
0927 23:28:01.279579 (+ 30us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":79308,"mutex_wait_us":574,"prepare.queue_time_us":191,"prepare.run_cpu_time_us":83756,"prepare.run_wall_time_us":215675,"raft.queue_time_us":4152,"raft.run_cpu_time_us":2141,"raft.run_wall_time_us":55591,"replication_time_us":731221,"spinlock_wait_cycles":1879680,"thread_start_us":259709,"threads_started":2,"wal-append.queue_time_us":303252}]]}
I20260927 23:28:01.317440 2971 raft_consensus.cc:2454] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:01.294173 3072 tablet_bootstrap.cc:654] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:01.320664 2608 leader_election.cc:302] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:01.322144 3112 raft_consensus.cc:2781] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:01.323115 3112 raft_consensus.cc:686] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:01.324200 3112 consensus_queue.cc:237] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.333840 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66f2842218fc42e5b1b83314e5f1a158" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:01.336050 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "828f870365244777b59e8a0756d7d4c9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:01.337564 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3acdfe7faa1a4d5d8e340f09f1cae93e" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:01.338645 2827 raft_consensus.cc:2379] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5ce671f028a34ea4b8f0520c2689c9a1 in current term 1: Already voted for candidate de216a4a8e4b4e1b857d8647a3b05f6d in this term.
I20260927 23:28:01.351856 3070 tablet_bootstrap.cc:654] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:00.922994 3167 raft_consensus.cc:348] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.371141 3167 raft_consensus.cc:374] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:01.371533 3167 raft_consensus.cc:729] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:01.371531 3311 raft_consensus.cc:482] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:01.372331 3167 consensus_queue.cc:260] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.373698 3164 raft_consensus.cc:482] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:01.373417 3311 raft_consensus.cc:504] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.374305 3072 tablet_bootstrap.cc:492] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:01.374127 3164 raft_consensus.cc:504] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.374722 3072 ts_tablet_manager.cc:1397] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.201s user 0.004s sys 0.007s
I20260927 23:28:01.375988 3164 leader_election.cc:288] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:01.376919 3242 leader_election.cc:288] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:01.377159 3072 raft_consensus.cc:348] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.377831 3072 raft_consensus.cc:374] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:01.378083 3072 raft_consensus.cc:729] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:01.378618 3242 consensus_queue.cc:1035] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:01.383649 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ca42b47d8044adc99a82859bf79e606" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:01.387440 2672 raft_consensus.cc:3037] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:01.388677 3311 leader_election.cc:288] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:01.378674 3072 consensus_queue.cc:260] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.392450 3072 ts_tablet_manager.cc:1428] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.017s user 0.007s sys 0.001s
I20260927 23:28:01.393397 3072 tablet_bootstrap.cc:492] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:01.396823 2672 raft_consensus.cc:2454] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:28:01.398514 2763 leader_election.cc:302] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:01.399722 3255 raft_consensus.cc:2781] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election won for term 1
W20260927 23:28:01.401525 2828 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s user 0.022s sys 0.062s
W20260927 23:28:01.401950 2828 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.542s user 0.022s sys 0.062s
I20260927 23:28:01.401674 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac49c5b79e347158621dda52f79e912" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:01.400269 3255 raft_consensus.cc:686] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Becoming Leader. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Running, Role: LEADER
I20260927 23:28:01.404618 3255 consensus_queue.cc:237] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:28:01.406026 2907 leader_election.cc:341] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.406886 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9670df27c4404acda97661c69420b5cf" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:01.410985 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9670df27c4404acda97661c69420b5cf" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:01.411816 2907 leader_election.cc:341] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:01.414736 2906 leader_election.cc:341] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.415226 2906 leader_election.cc:302] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:01.416545 2906 leader_election.cc:341] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Not found: Tablet not found: fac49c5b79e347158621dda52f79e912
I20260927 23:28:01.417001 2906 leader_election.cc:302] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:01.417409 3072 tablet_bootstrap.cc:654] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:01.417752 3164 raft_consensus.cc:2726] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:01.418294 3164 raft_consensus.cc:2726] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:01.423983 3295 tablet.cc:1722] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260927 23:28:01.425279 2826 thread.cc:636] apply [worker] (thread pool) Time spent creating pthread: real 0.538s user 0.019s sys 0.058s
W20260927 23:28:01.425894 2826 thread.cc:602] apply [worker] (thread pool) Time spent starting thread: real 0.539s user 0.019s sys 0.059s
I20260927 23:28:01.443640 3312 raft_consensus.cc:482] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:01.444229 3312 raft_consensus.cc:504] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.446552 3312 leader_election.cc:288] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:01.448858 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc1c0da6ea5c4a09a71555f37becae33" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:01.448869 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc1c0da6ea5c4a09a71555f37becae33" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:01.450580 2673 raft_consensus.cc:2454] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
W20260927 23:28:01.450619 2906 leader_election.cc:341] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.451784 2907 leader_election.cc:302] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:01.453035 3312 raft_consensus.cc:2781] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:01.453401 3312 raft_consensus.cc:482] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:01.453790 3312 raft_consensus.cc:3037] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:01.454483 2803 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42492 (request call id 53) took 1438 ms. Trace:
I20260927 23:28:01.454905 2803 rpcz_store.cc:276] 0927 23:28:00.016162 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:00.016320 (+ 158us) service_pool.cc:227] Handling call
0927 23:28:00.260497 (+244177us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:00.260601 (+ 104us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:01.273520 (+1012919us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:01.284203 (+ 10683us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:01.454298 (+170095us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:01.467015 3312 raft_consensus.cc:504] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.477299 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc1c0da6ea5c4a09a71555f37becae33" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:01.478165 2673 raft_consensus.cc:3037] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:01.495395 3070 tablet_bootstrap.cc:492] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:01.496022 3070 ts_tablet_manager.cc:1397] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.204s user 0.008s sys 0.004s
I20260927 23:28:01.498446 3070 raft_consensus.cc:348] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.504112 3167 ts_tablet_manager.cc:1428] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.752s user 0.014s sys 0.007s
W20260927 23:28:01.504467 3167 ts_tablet_manager.cc:1460] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51: Tablet startup took 752ms
W20260927 23:28:01.504737 3167 ts_tablet_manager.cc:1462] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51: Trace:
0927 23:27:57.720604 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:57.720824 (+ 220us) service_pool.cc:227] Handling call
0927 23:27:57.740515 (+ 19691us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:27:57.740627 (+ 112us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:27:57.761461 (+ 20834us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:57.767366 (+ 5905us) tablet_replica.cc:205] Creating consensus instance
0927 23:27:57.798203 (+ 30837us) inbound_call.cc:173] Queueing success response
0927 23:28:00.203656 (+2405453us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0927 23:28:00.752155 (+548499us) ts_tablet_manager.cc:1429] Starting tablet replica
0927 23:28:00.753177 (+ 1022us) tablet_replica.cc:259] Starting instrumentation
0927 23:28:00.754373 (+ 1196us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":6977,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":174,"tablet-open.queue_time_us":2435934,"thread_start_us":129336,"threads_started":1}
I20260927 23:28:01.515038 3072 tablet_bootstrap.cc:492] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:01.515547 3072 ts_tablet_manager.cc:1397] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.122s user 0.021s sys 0.006s
I20260927 23:28:01.527325 2391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 3c80915269094d3e8554c1655a1c98af
I20260927 23:28:01.528756 2391 catalog_manager.cc:6565] ReplaceTablet: tablet 3c80915269094d3e8554c1655a1c98af deleted and replaced by tablet 3a4c099683d1426282efd580a552c97b
I20260927 23:28:01.518108 3072 raft_consensus.cc:348] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:01.529838 3072 raft_consensus.cc:374] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:01.530179 3072 raft_consensus.cc:729] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:01.530889 3072 consensus_queue.cc:260] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:01.519860 3312 leader_election.cc:288] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:28:01.501937 2609 leader_election.cc:341] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:01.533324 2609 leader_election.cc:341] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.534281 2609 leader_election.cc:302] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:01.535463 3112 raft_consensus.cc:2726] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260927 23:28:01.533201 2652 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c80915269094d3e8554c1655a1c98af with delete_type TABLET_DATA_DELETED (replaced by tablet 3a4c099683d1426282efd580a552c97b) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:01.506613 2673 raft_consensus.cc:2454] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:01.536923 3070 raft_consensus.cc:374] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:01.537204 3070 raft_consensus.cc:729] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:01.538460 2907 leader_election.cc:302] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:01.539546 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c80915269094d3e8554c1655a1c98af with delete_type TABLET_DATA_DELETED (replaced by tablet 3a4c099683d1426282efd580a552c97b) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:01.540839 3311 raft_consensus.cc:2781] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:01.542060 3311 raft_consensus.cc:686] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:01.546133 3072 ts_tablet_manager.cc:1428] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.030s user 0.006s sys 0.003s
I20260927 23:28:01.547142 3072 tablet_bootstrap.cc:492] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:01.544723 3311 consensus_queue.cc:237] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.549660 3336 tablet_replica.cc:330] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:01.551322 3336 raft_consensus.cc:2229] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:01.552964 3336 raft_consensus.cc:2258] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:01.552528 3070 consensus_queue.cc:260] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.560003 3336 ts_tablet_manager.cc:1905] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:01.563230 3072 tablet_bootstrap.cc:654] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:01.564685 3312 raft_consensus.cc:482] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:01.565647 3312 raft_consensus.cc:504] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.568743 3312 leader_election.cc:288] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:01.575506 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cce47b050b445dfba7be5ea3b6a8636" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:01.577169 2907 leader_election.cc:341] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.580950 3070 ts_tablet_manager.cc:1428] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.085s user 0.003s sys 0.003s
I20260927 23:28:01.589628 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 04f670db808c4e289c655679da9d2d9d (DEFAULT_TABLE table=default.Table_0cce1d1549d5401c8b6c9966fe2674f4 [id=bd8e999821b44f628e5ec3b1d7915773]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.590998 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04f670db808c4e289c655679da9d2d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.594640 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 410294c28c49404e9c547b27111712eb (table default.Table_2cc5b0c1598e4ff282bbf7668ef916f0 [id=b215370f7b594ab28c3e2a34de2a6f16]). Expected version 1 got 0
I20260927 23:28:01.595541 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (table default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]). Expected version 1 got 0
I20260927 23:28:01.596835 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cce47b050b445dfba7be5ea3b6a8636" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:01.598352 2906 leader_election.cc:341] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.598857 2906 leader_election.cc:302] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:01.599640 3312 raft_consensus.cc:2726] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:01.605306 2945 tablet_service.cc:1467] Processing CreateTablet for tablet 7767a32c07454fe6a92154429a3b6899 (DEFAULT_TABLE table=default.Table_6568f80896124508811607305bf18ff9 [id=adbf504950b64a08996f102f4ac2d662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.606132 3072 tablet_bootstrap.cc:492] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:01.588758 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 04f670db808c4e289c655679da9d2d9d (DEFAULT_TABLE table=default.Table_0cce1d1549d5401c8b6c9966fe2674f4 [id=bd8e999821b44f628e5ec3b1d7915773]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.606567 3072 ts_tablet_manager.cc:1397] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.060s user 0.010s sys 0.000s
I20260927 23:28:01.606573 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7767a32c07454fe6a92154429a3b6899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.607298 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04f670db808c4e289c655679da9d2d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.608999 3072 raft_consensus.cc:348] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:01.609642 3072 raft_consensus.cc:374] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:01.609858 3072 raft_consensus.cc:729] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:01.610555 3072 consensus_queue.cc:260] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:01.603394 2653 tablet_service.cc:1467] Processing CreateTablet for tablet 7767a32c07454fe6a92154429a3b6899 (DEFAULT_TABLE table=default.Table_6568f80896124508811607305bf18ff9 [id=adbf504950b64a08996f102f4ac2d662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.613118 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7767a32c07454fe6a92154429a3b6899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.613065 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 43096d258a654532906f59d581f5a5d6 (table default.Table_3777b02a50e04654b10cc6bd6c73f51e [id=d3de7e8bfbed4409bf10ead542fe1c9d]). Expected version 2 got 0
I20260927 23:28:01.614017 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]). Expected version 1 got 0
I20260927 23:28:01.614604 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 45dd55b5a9d94ff0b84a799174ab5359 (table default.Table_72f38013370a483784dfabf8e4646d24 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 1 got 0
I20260927 23:28:01.617905 2389 catalog_manager.cc:5495] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe182b2afc64d908f692ef45eae1d51 (127.2.57.2). New cstate: current_term: 1 leader_uuid: "abe182b2afc64d908f692ef45eae1d51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: HEALTHY } } }
I20260927 23:28:01.619638 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]). Expected version 1 got 0
I20260927 23:28:01.621364 3336 ts_tablet_manager.cc:1918] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:01.621822 3336 log.cc:1196] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/3c80915269094d3e8554c1655a1c98af
I20260927 23:28:01.622795 3336 ts_tablet_manager.cc:1939] T 3c80915269094d3e8554c1655a1c98af P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:01.623865 2648 tablet_service.cc:1467] Processing CreateTablet for tablet ef1b6b75dad24c609a10693728ba8e20 (DEFAULT_TABLE table=default.Table_f21bcb78bde641968c28039a3bf9c800 [id=daf1b6381ae047d192111faaa10f119d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.625136 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef1b6b75dad24c609a10693728ba8e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.625777 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 3c80915269094d3e8554c1655a1c98af (table default.Table_52009c39b67d47ea919822fecb9fcc5f [id=d4bab68cc19145baae5fab5691c88c8b]) successfully deleted
I20260927 23:28:01.632732 2947 tablet_service.cc:1467] Processing CreateTablet for tablet ef1b6b75dad24c609a10693728ba8e20 (DEFAULT_TABLE table=default.Table_f21bcb78bde641968c28039a3bf9c800 [id=daf1b6381ae047d192111faaa10f119d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.622943 3072 ts_tablet_manager.cc:1428] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260927 23:28:01.634070 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef1b6b75dad24c609a10693728ba8e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.635816 3072 tablet_bootstrap.cc:492] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:01.637780 3312 raft_consensus.cc:482] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:01.638242 3312 raft_consensus.cc:504] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:01.640188 3312 leader_election.cc:288] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:01.641433 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d37dba66b84d05abf675d3eaf39a30" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:01.644543 2907 leader_election.cc:341] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.650967 3070 tablet_bootstrap.cc:492] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:01.655043 2647 tablet_service.cc:1467] Processing CreateTablet for tablet 884b80a1d31349c98f75525416b3c527 (DEFAULT_TABLE table=default.Table_c632a6c33818436ca316f423eb2b3aa7 [id=52c6e4c418ed403792b4bcb9e4ff3458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.656344 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 884b80a1d31349c98f75525416b3c527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.658214 3072 tablet_bootstrap.cc:654] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:01.681295 3334 tablet.cc:1722] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:01.685258 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc1c0da6ea5c4a09a71555f37becae33" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:01.686930 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 04f670db808c4e289c655679da9d2d9d (DEFAULT_TABLE table=default.Table_0cce1d1549d5401c8b6c9966fe2674f4 [id=bd8e999821b44f628e5ec3b1d7915773]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.689316 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04f670db808c4e289c655679da9d2d9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.693638 3072 tablet_bootstrap.cc:492] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:01.694080 3072 ts_tablet_manager.cc:1397] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.058s user 0.006s sys 0.003s
I20260927 23:28:01.697075 3072 raft_consensus.cc:348] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:01.697765 3072 raft_consensus.cc:374] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:01.698017 3072 raft_consensus.cc:729] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:01.699061 3072 consensus_queue.cc:260] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
W20260927 23:28:01.701555 2906 leader_election.cc:341] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.704720 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66f2842218fc42e5b1b83314e5f1a158" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:01.680804 2803 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c80915269094d3e8554c1655a1c98af with delete_type TABLET_DATA_DELETED (replaced by tablet 3a4c099683d1426282efd580a552c97b) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:01.694096 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 7767a32c07454fe6a92154429a3b6899 (DEFAULT_TABLE table=default.Table_6568f80896124508811607305bf18ff9 [id=adbf504950b64a08996f102f4ac2d662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.713233 3312 raft_consensus.cc:482] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:01.713770 3312 raft_consensus.cc:504] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.714773 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d37dba66b84d05abf675d3eaf39a30" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:01.715907 3312 leader_election.cc:288] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:28:01.720374 2906 leader_election.cc:341] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.720542 3072 ts_tablet_manager.cc:1428] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.026s user 0.005s sys 0.000s
I20260927 23:28:01.720832 2906 leader_election.cc:302] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:01.721583 3072 tablet_bootstrap.cc:492] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:01.721549 3312 raft_consensus.cc:2726] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:01.722285 2807 tablet_service.cc:1467] Processing CreateTablet for tablet 884b80a1d31349c98f75525416b3c527 (DEFAULT_TABLE table=default.Table_c632a6c33818436ca316f423eb2b3aa7 [id=52c6e4c418ed403792b4bcb9e4ff3458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.723538 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7767a32c07454fe6a92154429a3b6899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.725133 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5efbd0211cf3408cb5ea9640d8da6694" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:01.710966 2393 catalog_manager.cc:5495] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:01.726343 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5efbd0211cf3408cb5ea9640d8da6694" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:01.726543 2393 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 43096d258a654532906f59d581f5a5d6 (table default.Table_3777b02a50e04654b10cc6bd6c73f51e [id=d3de7e8bfbed4409bf10ead542fe1c9d]). Expected version 2 got 0
W20260927 23:28:01.727797 2907 leader_election.cc:341] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:01.728358 2906 leader_election.cc:341] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:01.728662 2393 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]). Expected version 1 got 0
I20260927 23:28:01.728775 2906 leader_election.cc:302] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:01.729043 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 884b80a1d31349c98f75525416b3c527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.729504 3312 raft_consensus.cc:2726] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:01.733986 2609 leader_election.cc:341] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:01.742063 2609 consensus_peers.cc:487] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:01.742980 2822 raft_consensus.cc:3037] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:01.753043 3072 tablet_bootstrap.cc:654] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:01.748926 2804 tablet_service.cc:1467] Processing CreateTablet for tablet ef1b6b75dad24c609a10693728ba8e20 (DEFAULT_TABLE table=default.Table_f21bcb78bde641968c28039a3bf9c800 [id=daf1b6381ae047d192111faaa10f119d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.754727 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ef1b6b75dad24c609a10693728ba8e20. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.753281 2393 catalog_manager.cc:5495] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:01.769935 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 884b80a1d31349c98f75525416b3c527 (DEFAULT_TABLE table=default.Table_c632a6c33818436ca316f423eb2b3aa7 [id=52c6e4c418ed403792b4bcb9e4ff3458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.771216 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 884b80a1d31349c98f75525416b3c527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.774340 3072 tablet_bootstrap.cc:492] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:01.774777 3072 ts_tablet_manager.cc:1397] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.053s user 0.004s sys 0.008s
I20260927 23:28:01.775017 3341 tablet_replica.cc:330] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:01.775918 3341 raft_consensus.cc:2229] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:01.776499 3341 raft_consensus.cc:2258] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:01.777158 3072 raft_consensus.cc:348] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:01.777989 3072 raft_consensus.cc:374] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:01.778249 3072 raft_consensus.cc:729] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:01.778810 3072 consensus_queue.cc:260] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:01.788066 3342 tablet.cc:1722] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:01.797549 3072 ts_tablet_manager.cc:1428] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.022s user 0.006s sys 0.001s
I20260927 23:28:01.833950 2803 tablet_service.cc:1467] Processing CreateTablet for tablet a4277045ec344ad7a48db9b14bea5e7b (DEFAULT_TABLE table=default.Table_a902e331f690448a872f6c104857f2ca [id=1acae7217ece44919a8d247f5cf155d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.839432 3112 consensus_queue.cc:1035] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:01.844970 3335 tablet_replica.cc:330] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:01.846334 3070 tablet_bootstrap.cc:654] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:01.846257 3335 raft_consensus.cc:2229] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:01.847968 3335 raft_consensus.cc:2258] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:01.852221 3112 raft_consensus.cc:482] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:01.852686 3112 raft_consensus.cc:504] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.855835 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d299ca52cfc493499b5eaf0acdeb5f9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:01.855949 3112 leader_election.cc:288] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:01.856676 2970 raft_consensus.cc:2454] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:01.858009 2608 leader_election.cc:302] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:01.859400 3112 raft_consensus.cc:2781] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:01.794065 2393 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (table default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]). Expected version 1 got 0
I20260927 23:28:01.859839 3112 raft_consensus.cc:482] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:01.860226 3112 raft_consensus.cc:3037] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:01.860235 2393 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 45dd55b5a9d94ff0b84a799174ab5359 (table default.Table_72f38013370a483784dfabf8e4646d24 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 1 got 0
I20260927 23:28:01.882776 3072 tablet_bootstrap.cc:492] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:01.887159 3112 raft_consensus.cc:504] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:01.900457 3072 tablet_bootstrap.cc:654] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:01.903079 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae1e85c32a94f868ef48ad471462b3e (DEFAULT_TABLE table=default.Table_9088c5760dc44415bf9a9106a40f5ab4 [id=21118fc25eb9485b92a9034bbeef8b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.904428 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae1e85c32a94f868ef48ad471462b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.887815 3335 ts_tablet_manager.cc:1905] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:01.912840 3112 leader_election.cc:288] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:01.925241 2646 tablet_service.cc:1467] Processing CreateTablet for tablet a4277045ec344ad7a48db9b14bea5e7b (DEFAULT_TABLE table=default.Table_a902e331f690448a872f6c104857f2ca [id=1acae7217ece44919a8d247f5cf155d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:01.926756 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4277045ec344ad7a48db9b14bea5e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.937986 3335 ts_tablet_manager.cc:1918] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:01.944001 3335 log.cc:1196] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/3c80915269094d3e8554c1655a1c98af
I20260927 23:28:01.945331 3335 ts_tablet_manager.cc:1939] T 3c80915269094d3e8554c1655a1c98af P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:01.955384 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 3c80915269094d3e8554c1655a1c98af (table default.Table_52009c39b67d47ea919822fecb9fcc5f [id=d4bab68cc19145baae5fab5691c88c8b]) successfully deleted
I20260927 23:28:01.966602 2387 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (table default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]). Expected version 1 got 0
I20260927 23:28:01.967631 2387 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 43096d258a654532906f59d581f5a5d6 (table default.Table_3777b02a50e04654b10cc6bd6c73f51e [id=d3de7e8bfbed4409bf10ead542fe1c9d]). Expected version 2 got 0
I20260927 23:28:01.968349 2387 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]). Expected version 1 got 0
I20260927 23:28:01.969051 2387 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 45dd55b5a9d94ff0b84a799174ab5359 (table default.Table_72f38013370a483784dfabf8e4646d24 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 1 got 0
I20260927 23:28:01.970119 2392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 39b3bf40c51646d38559f6852ebad6f0
I20260927 23:28:01.974041 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 39b3bf40c51646d38559f6852ebad6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:01.971846 2392 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 96) took 2263 ms. Trace:
I20260927 23:28:01.974887 3335 tablet_replica.cc:330] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:01.975772 3335 raft_consensus.cc:2229] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:01.976902 3335 raft_consensus.cc:2258] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:01.974515 2392 rpcz_store.cc:276] 0927 23:27:59.708144 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:59.708278 (+ 134us) service_pool.cc:227] Handling call
0927 23:27:59.709810 (+ 1532us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:27:59.710400 (+ 590us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:27:59.710567 (+ 167us) catalog_manager.cc:2761] Locking tablets
0927 23:27:59.711174 (+ 607us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:27:59.893647 (+182473us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:27:59.893890 (+ 243us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:01.970005 (+2076115us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:01.971764 (+ 1759us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:27:59.721896 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:27:59.722268 (+ 372us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:27:59.722292 (+ 24us) write_op.cc:435] Acquired schema lock
0927 23:27:59.722342 (+ 50us) tablet.cc:660] Decoding operations
0927 23:27:59.722693 (+ 351us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:27:59.722737 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0927 23:27:59.722762 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:27:59.723052 (+ 290us) tablet.cc:699] Row locks acquired
0927 23:27:59.723068 (+ 16us) write_op.cc:260] PREPARE: finished
0927 23:27:59.723199 (+ 131us) write_op.cc:270] Start()
0927 23:27:59.723302 (+ 103us) write_op.cc:276] Timestamp: P: 1790551679723110 usec, L: 0
0927 23:27:59.723328 (+ 26us) op_driver.cc:331] REPLICATION: starting
0927 23:27:59.723926 (+ 598us) log.cc:842] Serialized 721 byte log entry
0927 23:27:59.878805 (+154879us) op_driver.cc:444] REPLICATION: finished
0927 23:27:59.884470 (+ 5665us) write_op.cc:301] APPLY: starting
0927 23:27:59.884525 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0927 23:27:59.884631 (+ 106us) tablet.cc:1368] finished BulkCheckPresence
0927 23:27:59.884746 (+ 115us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:27:59.884957 (+ 211us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:27:59.885087 (+ 130us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:27:59.885130 (+ 43us) write_op.cc:312] APPLY: finished
0927 23:27:59.885311 (+ 181us) log.cc:842] Serialized 28 byte log entry
0927 23:27:59.885437 (+ 126us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:27:59.885490 (+ 53us) write_op.cc:454] Released schema lock
0927 23:27:59.885522 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0927 23:27:59.885684 (+ 162us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:27:59.885706 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5544,"apply.run_cpu_time_us":1426,"apply.run_wall_time_us":1426,"num_ops":2,"prepare.queue_time_us":8503,"prepare.run_cpu_time_us":2354,"prepare.run_wall_time_us":4415,"raft.queue_time_us":93,"raft.run_cpu_time_us":16,"raft.run_wall_time_us":13,"replication_time_us":155408}]]}
I20260927 23:28:01.980126 3335 ts_tablet_manager.cc:1905] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:01.988773 2391 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]). Expected version 1 got 0
I20260927 23:28:01.989454 2391 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 45dd55b5a9d94ff0b84a799174ab5359 (table default.Table_72f38013370a483784dfabf8e4646d24 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 1 got 0
I20260927 23:28:01.990010 2391 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 410294c28c49404e9c547b27111712eb (table default.Table_2cc5b0c1598e4ff282bbf7668ef916f0 [id=b215370f7b594ab28c3e2a34de2a6f16]). Expected version 1 got 0
I20260927 23:28:01.990839 2390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 2572268ca47346b0b78abc6a9ca7f908
W20260927 23:28:01.991047 2386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 3acdfe7faa1a4d5d8e340f09f1cae93e consensus state has no leader
I20260927 23:28:01.992471 2390 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 91) took 2525 ms. Trace:
W20260927 23:28:01.996114 2389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 26) took 2407 ms (client timeout 1000 ms). Trace:
I20260927 23:28:01.999188 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 2572268ca47346b0b78abc6a9ca7f908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:01.996802 2389 rpcz_store.cc:269] 0927 23:27:59.588820 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:59.600490 (+ 11670us) service_pool.cc:227] Handling call
0927 23:28:01.995974 (+2395484us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:01.629879 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:01.630266 (+ 387us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:01.630293 (+ 27us) write_op.cc:435] Acquired schema lock
0927 23:28:01.630308 (+ 15us) tablet.cc:660] Decoding operations
0927 23:28:01.630616 (+ 308us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:01.630663 (+ 47us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:01.630687 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:01.630845 (+ 158us) tablet.cc:699] Row locks acquired
0927 23:28:01.630870 (+ 25us) write_op.cc:260] PREPARE: finished
0927 23:28:01.630994 (+ 124us) write_op.cc:270] Start()
0927 23:28:01.631082 (+ 88us) write_op.cc:276] Timestamp: P: 1790551681630924 usec, L: 0
0927 23:28:01.631099 (+ 17us) op_driver.cc:331] REPLICATION: starting
0927 23:28:01.631609 (+ 510us) log.cc:842] Serialized 524 byte log entry
0927 23:28:01.680593 (+ 48984us) op_driver.cc:444] REPLICATION: finished
0927 23:28:01.687941 (+ 7348us) write_op.cc:301] APPLY: starting
0927 23:28:01.688009 (+ 68us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:01.688103 (+ 94us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:01.688203 (+ 100us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:01.689393 (+ 1190us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:01.689536 (+ 143us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:01.689575 (+ 39us) write_op.cc:312] APPLY: finished
0927 23:28:01.689789 (+ 214us) log.cc:842] Serialized 23 byte log entry
0927 23:28:01.690644 (+ 855us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:01.690719 (+ 75us) write_op.cc:454] Released schema lock
0927 23:28:01.690751 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:01.722261 (+ 31510us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:01.722297 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":7139,"apply.run_cpu_time_us":1761,"apply.run_wall_time_us":34590,"num_ops":1,"prepare.queue_time_us":135,"prepare.run_cpu_time_us":2080,"prepare.run_wall_time_us":2079,"raft.queue_time_us":2056,"raft.run_cpu_time_us":19,"raft.run_wall_time_us":21,"replication_time_us":49386}]]}
I20260927 23:28:02.016888 3335 ts_tablet_manager.cc:1918] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:02.017454 3335 log.cc:1196] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/39b3bf40c51646d38559f6852ebad6f0
W20260927 23:28:02.018092 2380 catalog_manager.cc:4914] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): alter failed for tablet 3c80915269094d3e8554c1655a1c98af (table default.Table_52009c39b67d47ea919822fecb9fcc5f [id=d4bab68cc19145baae5fab5691c88c8b]),no further retry: Not found: Tablet not found: 3c80915269094d3e8554c1655a1c98af
I20260927 23:28:02.018378 3335 ts_tablet_manager.cc:1939] T 39b3bf40c51646d38559f6852ebad6f0 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
W20260927 23:28:02.018538 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 120047us
I20260927 23:28:02.014997 3311 consensus_queue.cc:1035] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260927 23:28:02.021787 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 39b3bf40c51646d38559f6852ebad6f0 (table default.Table_faba7f79481947c1a44139eddeb749a6 [id=2aa2af3d524a455e8d22c0171fe10942]) successfully deleted
I20260927 23:28:02.022053 3164 consensus_queue.cc:1035] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:02.023005 3335 tablet_replica.cc:330] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:02.024178 3335 raft_consensus.cc:2229] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:02.025218 3335 raft_consensus.cc:2258] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:02.014528 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 121309us
I20260927 23:28:02.030841 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d299ca52cfc493499b5eaf0acdeb5f9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:02.031504 2972 raft_consensus.cc:3037] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:02.035243 2385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 13) took 2683 ms (client timeout 1000 ms). Trace:
W20260927 23:28:02.035643 2385 rpcz_store.cc:269] 0927 23:27:59.352027 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:59.352194 (+ 167us) service_pool.cc:227] Handling call
0927 23:28:02.035136 (+2682942us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":7}
I20260927 23:28:02.039171 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet 517d7bfc225a4372bfd79682a550a0d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:02.033550 3335 ts_tablet_manager.cc:1905] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:01.992816 2390 rpcz_store.cc:276] 0927 23:27:59.466774 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:59.482352 (+ 15578us) service_pool.cc:227] Handling call
0927 23:27:59.514852 (+ 32500us) spinlock_profiling.cc:243] Waited 31.1 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b3466 00007f8bb3bf4b34 00007f8bb3bf4ac2 00007f8bb3bf49f1 00007f8bb3bf496c 00007f8bb3bf3c22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:27:59.516357 (+ 1505us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:27:59.516887 (+ 530us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:27:59.517051 (+ 164us) catalog_manager.cc:2761] Locking tablets
0927 23:27:59.746898 (+229847us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:28:00.445239 (+698341us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:28:00.446885 (+ 1646us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:01.990735 (+1543850us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:01.992385 (+ 1650us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:27:59.758460 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:27:59.866695 (+108235us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:27:59.866722 (+ 27us) write_op.cc:435] Acquired schema lock
0927 23:27:59.866745 (+ 23us) tablet.cc:660] Decoding operations
0927 23:27:59.867132 (+ 387us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:27:59.867172 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0927 23:27:59.867193 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:27:59.867480 (+ 287us) tablet.cc:699] Row locks acquired
0927 23:27:59.867494 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:27:59.867625 (+ 131us) write_op.cc:270] Start()
0927 23:27:59.867721 (+ 96us) write_op.cc:276] Timestamp: P: 1790551679867573 usec, L: 0
0927 23:27:59.867748 (+ 27us) op_driver.cc:331] REPLICATION: starting
0927 23:27:59.868387 (+ 639us) log.cc:842] Serialized 786 byte log entry
0927 23:28:00.418967 (+550580us) op_driver.cc:444] REPLICATION: finished
0927 23:28:00.437875 (+ 18908us) write_op.cc:301] APPLY: starting
0927 23:28:00.437942 (+ 67us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:00.438052 (+ 110us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:00.438174 (+ 122us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:00.438444 (+ 270us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:00.438591 (+ 147us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:00.438632 (+ 41us) write_op.cc:312] APPLY: finished
0927 23:28:00.438852 (+ 220us) log.cc:842] Serialized 28 byte log entry
0927 23:28:00.439019 (+ 167us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:00.439092 (+ 73us) write_op.cc:454] Released schema lock
0927 23:28:00.439126 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:00.445526 (+ 6400us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:00.445580 (+ 54us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":71528320,"child_traces":[["op",{"apply.queue_time_us":18680,"apply.run_cpu_time_us":1905,"apply.run_wall_time_us":8057,"num_ops":2,"prepare.queue_time_us":9283,"prepare.run_cpu_time_us":2427,"prepare.run_wall_time_us":111555,"replication_time_us":480806,"thread_start_us":62263,"threads_started":1}]]}
I20260927 23:28:02.041105 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae1e85c32a94f868ef48ad471462b3e (DEFAULT_TABLE table=default.Table_9088c5760dc44415bf9a9106a40f5ab4 [id=21118fc25eb9485b92a9034bbeef8b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:02.042356 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae1e85c32a94f868ef48ad471462b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:02.048940 3112 consensus_queue.cc:1035] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:02.050464 2385 master_service.cc:946] ReplaceTablet: received request to replace tablet 828f870365244777b59e8a0756d7d4c9 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:02.042837 2951 tablet_service.cc:1514] Processing DeleteTablet for tablet 39b3bf40c51646d38559f6852ebad6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:02.058142 3336 tablet_replica.cc:330] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:02.059130 3336 raft_consensus.cc:2229] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:02.059715 3336 raft_consensus.cc:2258] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:02.043381 2944 tablet_service.cc:1514] Processing DeleteTablet for tablet 2572268ca47346b0b78abc6a9ca7f908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:02.060672 2386 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:51972 (request call id 95) took 2369 ms. Trace:
I20260927 23:28:02.062813 3336 ts_tablet_manager.cc:1905] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:02.065338 2384 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:51972 (request call id 92) took 2559 ms. Trace:
W20260927 23:28:02.068660 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 517d7bfc225a4372bfd79682a550a0d1 because the tablet was not found. No further retry: Not found: Tablet not found: 517d7bfc225a4372bfd79682a550a0d1
I20260927 23:28:02.068876 2390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_678fca7e3fe641c39d8cdc436fc85d38"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:02.051385 2946 tablet_service.cc:1467] Processing CreateTablet for tablet a4277045ec344ad7a48db9b14bea5e7b (DEFAULT_TABLE table=default.Table_a902e331f690448a872f6c104857f2ca [id=1acae7217ece44919a8d247f5cf155d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:02.072104 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4277045ec344ad7a48db9b14bea5e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:02.065847 2384 rpcz_store.cc:276] 0927 23:27:59.506377 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:59.539961 (+ 33584us) service_pool.cc:227] Handling call
0927 23:27:59.541348 (+ 1387us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:27:59.542271 (+ 923us) catalog_manager.cc:3616] Apply alter schema
0927 23:27:59.543206 (+ 935us) catalog_manager.cc:3639] Acquired catalog manager lock
0927 23:27:59.543437 (+ 231us) catalog_manager.cc:3786] Updating metadata on disk
0927 23:28:01.090990 (+1547553us) catalog_manager.cc:3817] Committing alterations to in-memory state
0927 23:28:02.065258 (+974268us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:00.385776 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:00.386252 (+ 476us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:00.386279 (+ 27us) write_op.cc:435] Acquired schema lock
0927 23:28:00.386293 (+ 14us) tablet.cc:660] Decoding operations
0927 23:28:00.386575 (+ 282us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:00.386613 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:00.386636 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:00.386795 (+ 159us) tablet.cc:699] Row locks acquired
0927 23:28:00.386809 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:00.386936 (+ 127us) write_op.cc:270] Start()
0927 23:28:00.387025 (+ 89us) write_op.cc:276] Timestamp: P: 1790551680386896 usec, L: 0
0927 23:28:00.387040 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:00.549558 (+162518us) log.cc:842] Serialized 404 byte log entry
0927 23:28:00.552942 (+ 3384us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068471 00007f8bb40df198 00007f8bb4540ade 00007f8bb45402b8 00007f8bb4544ba1 00007f8bb4544b59 00007f8bb4544ae9 00007f8bb4544ab1 00007f8bb4543dad 0000000000580784 00000000005805b9
0927 23:28:01.087227 (+534285us) op_driver.cc:444] REPLICATION: finished
0927 23:28:01.087973 (+ 746us) write_op.cc:301] APPLY: starting
0927 23:28:01.088046 (+ 73us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:01.088158 (+ 112us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:01.088176 (+ 18us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:01.088379 (+ 203us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:01.088512 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:01.088559 (+ 47us) write_op.cc:312] APPLY: finished
0927 23:28:01.088751 (+ 192us) log.cc:842] Serialized 22 byte log entry
0927 23:28:01.088932 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:01.089001 (+ 69us) write_op.cc:454] Released schema lock
0927 23:28:01.089032 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:01.089212 (+ 180us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:01.089235 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":565,"apply.run_cpu_time_us":1437,"apply.run_wall_time_us":1437,"num_ops":1,"prepare.queue_time_us":435362,"prepare.run_cpu_time_us":3680,"prepare.run_wall_time_us":167619,"raft.queue_time_us":117,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":16,"replication_time_us":700088,"spinlock_wait_cycles":307001216}]]}
I20260927 23:28:02.080391 2384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_63f9354acd7d4ca3ad92ab1edf9d7174"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:02.099920 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 39b3bf40c51646d38559f6852ebad6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:02.061064 2386 rpcz_store.cc:276] 0927 23:27:59.691419 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:59.691543 (+ 124us) service_pool.cc:227] Handling call
0927 23:27:59.692922 (+ 1379us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:27:59.693661 (+ 739us) catalog_manager.cc:3616] Apply alter schema
0927 23:27:59.694403 (+ 742us) catalog_manager.cc:3639] Acquired catalog manager lock
0927 23:27:59.694633 (+ 230us) catalog_manager.cc:3786] Updating metadata on disk
0927 23:27:59.894080 (+199447us) catalog_manager.cc:3817] Committing alterations to in-memory state
0927 23:28:02.060577 (+2166497us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:27:59.698727 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:27:59.699101 (+ 374us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:27:59.699122 (+ 21us) write_op.cc:435] Acquired schema lock
0927 23:27:59.699136 (+ 14us) tablet.cc:660] Decoding operations
0927 23:27:59.699429 (+ 293us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:27:59.699467 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0927 23:27:59.699488 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:27:59.699643 (+ 155us) tablet.cc:699] Row locks acquired
0927 23:27:59.699654 (+ 11us) write_op.cc:260] PREPARE: finished
0927 23:27:59.699786 (+ 132us) write_op.cc:270] Start()
0927 23:27:59.699876 (+ 90us) write_op.cc:276] Timestamp: P: 1790551679699701 usec, L: 0
0927 23:27:59.699892 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:27:59.700474 (+ 582us) log.cc:842] Serialized 404 byte log entry
0927 23:27:59.878605 (+178131us) op_driver.cc:444] REPLICATION: finished
0927 23:27:59.882976 (+ 4371us) write_op.cc:301] APPLY: starting
0927 23:27:59.883031 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0927 23:27:59.883126 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0927 23:27:59.883225 (+ 99us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:27:59.883366 (+ 141us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:27:59.883487 (+ 121us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:27:59.883527 (+ 40us) write_op.cc:312] APPLY: finished
0927 23:27:59.883696 (+ 169us) log.cc:842] Serialized 22 byte log entry
0927 23:27:59.883845 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:27:59.883900 (+ 55us) write_op.cc:454] Released schema lock
0927 23:27:59.883932 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0927 23:27:59.884130 (+ 198us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:27:59.884155 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4219,"apply.run_cpu_time_us":1354,"apply.run_wall_time_us":1354,"num_ops":1,"prepare.queue_time_us":2792,"prepare.run_cpu_time_us":2046,"prepare.run_wall_time_us":2047,"raft.queue_time_us":1695,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":14,"replication_time_us":178650}]]}
W20260927 23:28:02.089097 2390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_678fca7e3fe641c39d8cdc436fc85d38 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:02.101380 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_2d9f46546b8a45fd8747186ee6775db1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:02.101380 2388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 20) took 2593 ms (client timeout 1000 ms). Trace:
W20260927 23:28:02.115991 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2d9f46546b8a45fd8747186ee6775db1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:02.115908 2388 rpcz_store.cc:269] 0927 23:27:59.507829 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:59.507977 (+ 148us) service_pool.cc:227] Handling call
0927 23:27:59.517986 (+ 10009us) spinlock_profiling.cc:243] Waited 6.95 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:02.101284 (+2583298us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":58820,"reported_tablets":8,"spinlock_wait_cycles":15992192}
I20260927 23:28:02.118131 3335 ts_tablet_manager.cc:1918] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:02.119936 3335 log.cc:1196] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/2572268ca47346b0b78abc6a9ca7f908
I20260927 23:28:02.120790 2647 tablet_service.cc:1467] Processing CreateTablet for tablet d47c2f22abbe40e7ba8ea945ab96aeaf (DEFAULT_TABLE table=default.Table_0c873b78624f41d18d0d82e10a46d738 [id=c8c0a7367a8342afb8ffc95627013a9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:02.123226 3335 ts_tablet_manager.cc:1939] T 2572268ca47346b0b78abc6a9ca7f908 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
W20260927 23:28:02.123660 2384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_63f9354acd7d4ca3ad92ab1edf9d7174 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:02.123314 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d47c2f22abbe40e7ba8ea945ab96aeaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:01.891129 3070 tablet_bootstrap.cc:492] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:02.126024 3070 ts_tablet_manager.cc:1397] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.475s user 0.011s sys 0.000s
I20260927 23:28:02.126426 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_69a2163339974120a45253fcfdb43e98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:02.128545 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_69a2163339974120a45253fcfdb43e98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:02.134913 2388 master_service.cc:946] ReplaceTablet: received request to replace tablet 45dd55b5a9d94ff0b84a799174ab5359 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:02.137838 2944 tablet_service.cc:1467] Processing CreateTablet for tablet d47c2f22abbe40e7ba8ea945ab96aeaf (DEFAULT_TABLE table=default.Table_0c873b78624f41d18d0d82e10a46d738 [id=c8c0a7367a8342afb8ffc95627013a9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:02.139078 2944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d47c2f22abbe40e7ba8ea945ab96aeaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:02.139525 3336 ts_tablet_manager.cc:1918] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:02.140049 3336 log.cc:1196] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/39b3bf40c51646d38559f6852ebad6f0
I20260927 23:28:02.140852 3336 ts_tablet_manager.cc:1939] T 39b3bf40c51646d38559f6852ebad6f0 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:02.142952 3112 consensus_queue.cc:1035] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:02.143846 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 39b3bf40c51646d38559f6852ebad6f0 (table default.Table_faba7f79481947c1a44139eddeb749a6 [id=2aa2af3d524a455e8d22c0171fe10942]) successfully deleted
I20260927 23:28:02.086809 2972 raft_consensus.cc:2454] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:02.148953 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 2572268ca47346b0b78abc6a9ca7f908 (table default.Table_2d889ed8e18e47a79f0541db93242386 [id=103d7c5e210c48fdafb8c199eb4cf676]) successfully deleted
I20260927 23:28:02.169771 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 2572268ca47346b0b78abc6a9ca7f908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:02.171149 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c80915269094d3e8554c1655a1c98af with delete_type TABLET_DATA_DELETED (replaced by tablet 3a4c099683d1426282efd580a552c97b) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:02.172847 2379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e (table default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 3acdfe7faa1a4d5d8e340f09f1cae93e consensus state has no leader
I20260927 23:28:02.178200 3070 raft_consensus.cc:348] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:02.178987 3070 raft_consensus.cc:374] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:02.179239 3070 raft_consensus.cc:729] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:02.180183 3070 consensus_queue.cc:260] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:02.184599 3070 ts_tablet_manager.cc:1428] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.058s user 0.007s sys 0.000s
I20260927 23:28:02.197589 2385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 828f870365244777b59e8a0756d7d4c9
I20260927 23:28:02.199079 2385 catalog_manager.cc:6565] ReplaceTablet: tablet 828f870365244777b59e8a0756d7d4c9 deleted and replaced by tablet 5bf856a05dff4fb4943fe8e1d9b7f385
I20260927 23:28:02.199481 2385 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:51972 (request call id 103) took 1226 ms. Trace:
I20260927 23:28:02.200580 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 828f870365244777b59e8a0756d7d4c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 5bf856a05dff4fb4943fe8e1d9b7f385) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:02.203871 2392 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 410294c28c49404e9c547b27111712eb (table default.Table_813e5d484f8f462b86e9136a57eebacd [id=b215370f7b594ab28c3e2a34de2a6f16]). Expected version 2 got 1
W20260927 23:28:02.205821 2393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 14) took 1772 ms (client timeout 1000 ms). Trace:
I20260927 23:28:02.208814 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4277045ec344ad7a48db9b14bea5e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:02.172892 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d299ca52cfc493499b5eaf0acdeb5f9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:02.226910 3112 consensus_queue.cc:1035] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:02.228387 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 1ae1e85c32a94f868ef48ad471462b3e (DEFAULT_TABLE table=default.Table_9088c5760dc44415bf9a9106a40f5ab4 [id=21118fc25eb9485b92a9034bbeef8b13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:02.240407 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ae1e85c32a94f868ef48ad471462b3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:02.245177 3070 tablet_bootstrap.cc:492] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:02.258069 3341 ts_tablet_manager.cc:1905] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:02.263259 2390 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 104) took 1172 ms. Trace:
I20260927 23:28:02.265373 3167 tablet_bootstrap.cc:492] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:02.263676 2390 rpcz_store.cc:276] 0927 23:28:01.090767 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.049312 (+958545us) service_pool.cc:227] Handling call
0927 23:28:02.119370 (+ 70058us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:02.130219 (+ 10849us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:02.262237 (+132018us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:02.262686 (+ 449us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:02.263172 (+ 486us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:02.145755 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:02.146122 (+ 367us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:02.146146 (+ 24us) write_op.cc:435] Acquired schema lock
0927 23:28:02.146160 (+ 14us) tablet.cc:660] Decoding operations
0927 23:28:02.146442 (+ 282us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:02.146476 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:02.146499 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:02.146705 (+ 206us) tablet.cc:699] Row locks acquired
0927 23:28:02.146721 (+ 16us) write_op.cc:260] PREPARE: finished
0927 23:28:02.146844 (+ 123us) write_op.cc:270] Start()
0927 23:28:02.146934 (+ 90us) write_op.cc:276] Timestamp: P: 1790551682146775 usec, L: 0
0927 23:28:02.146951 (+ 17us) op_driver.cc:331] REPLICATION: starting
0927 23:28:02.147466 (+ 515us) log.cc:842] Serialized 433 byte log entry
0927 23:28:02.238359 (+ 90893us) op_driver.cc:444] REPLICATION: finished
0927 23:28:02.240849 (+ 2490us) write_op.cc:301] APPLY: starting
0927 23:28:02.240900 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:02.241008 (+ 108us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:02.241119 (+ 111us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:02.242075 (+ 956us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:02.242226 (+ 151us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:02.242272 (+ 46us) write_op.cc:312] APPLY: finished
0927 23:28:02.242485 (+ 213us) log.cc:842] Serialized 29 byte log entry
0927 23:28:02.242662 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:02.242729 (+ 67us) write_op.cc:454] Released schema lock
0927 23:28:02.242771 (+ 42us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2345,"mutex_wait_us":13641,"num_ops":2,"prepare.queue_time_us":14090,"prepare.run_cpu_time_us":1925,"prepare.run_wall_time_us":2940,"replication_time_us":91336,"threads_started":1}]]}
I20260927 23:28:02.213094 2392 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:02.270303 2392 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 43096d258a654532906f59d581f5a5d6 (table default.Table_3777b02a50e04654b10cc6bd6c73f51e [id=d3de7e8bfbed4409bf10ead542fe1c9d]). Expected version 2 got 0
I20260927 23:28:02.271464 2392 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (table default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]). Expected version 1 got 0
I20260927 23:28:02.272173 2392 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 45dd55b5a9d94ff0b84a799174ab5359 (table default.Table_72f38013370a483784dfabf8e4646d24 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 1 got 0
I20260927 23:28:02.272972 2392 catalog_manager.cc:5495] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:02.242924 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d299ca52cfc493499b5eaf0acdeb5f9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:02.231909 2387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 21) took 1573 ms (client timeout 1000 ms). Trace:
I20260927 23:28:02.199798 2385 rpcz_store.cc:276] 0927 23:28:00.973606 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.048626 (+1075020us) service_pool.cc:227] Handling call
0927 23:28:02.199391 (+150765us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:02.107536 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:02.107955 (+ 419us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:02.107984 (+ 29us) write_op.cc:435] Acquired schema lock
0927 23:28:02.107997 (+ 13us) tablet.cc:660] Decoding operations
0927 23:28:02.108351 (+ 354us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:02.108396 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:02.108420 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:02.108625 (+ 205us) tablet.cc:699] Row locks acquired
0927 23:28:02.108639 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:02.108887 (+ 248us) write_op.cc:270] Start()
0927 23:28:02.108998 (+ 111us) write_op.cc:276] Timestamp: P: 1790551682108702 usec, L: 0
0927 23:28:02.109015 (+ 17us) op_driver.cc:331] REPLICATION: starting
0927 23:28:02.109639 (+ 624us) log.cc:842] Serialized 619 byte log entry
0927 23:28:02.191529 (+ 81890us) op_driver.cc:444] REPLICATION: finished
0927 23:28:02.191913 (+ 384us) write_op.cc:301] APPLY: starting
0927 23:28:02.191970 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:02.192089 (+ 119us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:02.192215 (+ 126us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:02.192622 (+ 407us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:02.192853 (+ 231us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:02.192903 (+ 50us) write_op.cc:312] APPLY: finished
0927 23:28:02.193125 (+ 222us) log.cc:842] Serialized 29 byte log entry
0927 23:28:02.193289 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:02.193366 (+ 77us) write_op.cc:454] Released schema lock
0927 23:28:02.193401 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:02.193591 (+ 190us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:02.193613 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":8576,"child_traces":[["op",{"apply.queue_time_us":210,"apply.run_cpu_time_us":1901,"apply.run_wall_time_us":1900,"num_ops":2,"prepare.queue_time_us":119,"prepare.run_cpu_time_us":2437,"prepare.run_wall_time_us":2436,"raft.queue_time_us":1425,"raft.run_cpu_time_us":772,"raft.run_wall_time_us":771,"replication_time_us":82419}]]}
W20260927 23:28:02.284972 2387 rpcz_store.cc:269] 0927 23:28:00.658187 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:01.092479 (+434292us) service_pool.cc:227] Handling call
0927 23:28:01.101303 (+ 8824us) spinlock_profiling.cc:243] Waited 5.38 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:02.231808 (+1130505us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":77812,"reported_tablets":11,"spinlock_wait_cycles":12371200}
I20260927 23:28:02.289003 2392 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 9cce47b050b445dfba7be5ea3b6a8636 (table default.Table_746dd85f958f4346bb7c3484f59e75f4 [id=b12aae65f933476ab57974d4b328a266]). Expected version 1 got 0
I20260927 23:28:02.290113 2392 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]). Expected version 1 got 0
I20260927 23:28:02.293473 2806 tablet_service.cc:1467] Processing CreateTablet for tablet d47c2f22abbe40e7ba8ea945ab96aeaf (DEFAULT_TABLE table=default.Table_0c873b78624f41d18d0d82e10a46d738 [id=c8c0a7367a8342afb8ffc95627013a9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:02.294653 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d47c2f22abbe40e7ba8ea945ab96aeaf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:02.297086 3070 tablet_bootstrap.cc:654] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:02.301162 2802 tablet_service.cc:1514] Processing DeleteTablet for tablet 39b3bf40c51646d38559f6852ebad6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:02.301968 2800 tablet_service.cc:1514] Processing DeleteTablet for tablet 2572268ca47346b0b78abc6a9ca7f908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:02.303349 2391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_a5f98162932247a3ae2928ce03a691a4" } modify_external_catalogs: true
I20260927 23:28:02.303987 2391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_a5f98162932247a3ae2928ce03a691a4" } modify_external_catalogs: true
W20260927 23:28:02.253139 2393 rpcz_store.cc:269] 0927 23:28:00.433436 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:00.882280 (+448844us) service_pool.cc:227] Handling call
0927 23:28:01.960410 (+1078130us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:01.960442 (+ 32us) write_op.cc:454] Released schema lock
0927 23:28:02.205746 (+245304us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:01.933110 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:01.933509 (+ 399us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:01.933535 (+ 26us) write_op.cc:435] Acquired schema lock
0927 23:28:01.933548 (+ 13us) tablet.cc:660] Decoding operations
0927 23:28:01.933890 (+ 342us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:01.933936 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:01.933957 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:01.934223 (+ 266us) tablet.cc:699] Row locks acquired
0927 23:28:01.934238 (+ 15us) write_op.cc:260] PREPARE: finished
0927 23:28:01.934381 (+ 143us) write_op.cc:270] Start()
0927 23:28:01.934479 (+ 98us) write_op.cc:276] Timestamp: P: 1790551681934333 usec, L: 0
0927 23:28:01.934495 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:01.935046 (+ 551us) log.cc:842] Serialized 895 byte log entry
0927 23:28:01.945384 (+ 10338us) op_driver.cc:444] REPLICATION: finished
0927 23:28:01.945742 (+ 358us) write_op.cc:301] APPLY: starting
0927 23:28:01.945804 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:01.945915 (+ 111us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:01.946004 (+ 89us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:01.946236 (+ 232us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:01.946377 (+ 141us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:01.946431 (+ 54us) write_op.cc:312] APPLY: finished
0927 23:28:01.946624 (+ 193us) log.cc:842] Serialized 29 byte log entry
0927 23:28:01.946776 (+ 152us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:01.946850 (+ 74us) write_op.cc:454] Released schema lock
0927 23:28:01.946883 (+ 33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":15,"child_traces":[["op",{"apply.queue_time_us":190,"apply.run_cpu_time_us":1333,"apply.run_wall_time_us":1332,"num_ops":2,"prepare.queue_time_us":115,"prepare.run_cpu_time_us":2238,"prepare.run_wall_time_us":2239,"raft.queue_time_us":182,"raft.run_cpu_time_us":698,"raft.run_wall_time_us":698,"replication_time_us":10807}]]}
I20260927 23:28:02.307927 2798 tablet_service.cc:1514] Processing DeleteTablet for tablet 828f870365244777b59e8a0756d7d4c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 5bf856a05dff4fb4943fe8e1d9b7f385) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:02.310671 2799 tablet_service.cc:1514] Processing DeleteTablet for tablet 517d7bfc225a4372bfd79682a550a0d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:42492
W20260927 23:28:02.314764 2609 leader_election.cc:341] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:02.316763 2609 leader_election.cc:341] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:02.318317 2807 consensus_meta.cc:287] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51: Time spent flushing consensus metadata: real 0.513s user 0.008s sys 0.001s
W20260927 23:28:02.327600 3298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:28:02.328539 3298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:28:02.329246 3298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260927 23:28:02.307200 2390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_52009c39b67d47ea919822fecb9fcc5f" } modify_external_catalogs: true
I20260927 23:28:02.334829 2390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_52009c39b67d47ea919822fecb9fcc5f" } modify_external_catalogs: true
I20260927 23:28:02.306344 2387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_50e57468f6b742cdab9659eb6efde970" } new_table_name: "default.Table_2b44b865509b414ba9c0c667dd764097" modify_external_catalogs: true
I20260927 23:28:02.335767 2384 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 105) took 1038 ms. Trace:
I20260927 23:28:02.336134 2384 rpcz_store.cc:276] 0927 23:28:01.297333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.079715 (+782382us) service_pool.cc:227] Handling call
0927 23:28:02.124259 (+ 44544us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:02.135148 (+ 10889us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:02.334919 (+199771us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:02.335335 (+ 416us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:02.335693 (+ 358us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:02.148800 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:02.149150 (+ 350us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:02.149171 (+ 21us) write_op.cc:435] Acquired schema lock
0927 23:28:02.149183 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:02.149459 (+ 276us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:02.149496 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:02.149517 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:02.149700 (+ 183us) tablet.cc:699] Row locks acquired
0927 23:28:02.149714 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:02.149831 (+ 117us) write_op.cc:270] Start()
0927 23:28:02.149914 (+ 83us) write_op.cc:276] Timestamp: P: 1790551682149763 usec, L: 0
0927 23:28:02.149930 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:02.150461 (+ 531us) log.cc:842] Serialized 433 byte log entry
0927 23:28:02.301329 (+150868us) op_driver.cc:444] REPLICATION: finished
0927 23:28:02.303589 (+ 2260us) write_op.cc:301] APPLY: starting
0927 23:28:02.303655 (+ 66us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:02.303785 (+ 130us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:02.303919 (+ 134us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:02.304422 (+ 503us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:02.304576 (+ 154us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:02.304627 (+ 51us) write_op.cc:312] APPLY: finished
0927 23:28:02.304872 (+ 245us) log.cc:842] Serialized 29 byte log entry
0927 23:28:02.307639 (+ 2767us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:02.307730 (+ 91us) write_op.cc:454] Released schema lock
0927 23:28:02.307770 (+ 40us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2030,"mutex_wait_us":4007,"num_ops":2,"prepare.queue_time_us":7156,"prepare.run_cpu_time_us":1879,"prepare.run_wall_time_us":2930,"replication_time_us":151308}]]}
I20260927 23:28:02.303390 2392 catalog_manager.cc:5495] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:02.346096 2392 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e (table default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2]). Expected version 1 got 0
W20260927 23:28:02.346930 2609 consensus_peers.cc:487] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:02.328228 2386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_e028dc8270d4406bb436aa0b2caf6dfe" } new_table_name: "default.Table_bf18dd7dd7984680bf41384772c15e61" modify_external_catalogs: true
I20260927 23:28:02.383066 3167 tablet_bootstrap.cc:654] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:02.380350 3070 tablet_bootstrap.cc:492] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:02.384924 3070 ts_tablet_manager.cc:1397] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.140s user 0.011s sys 0.008s
W20260927 23:28:02.392350 3298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:28:02.392771 3298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:28:02.393121 3298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:28:02.393456 3298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:28:02.387575 3070 raft_consensus.cc:348] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:02.411409 3070 raft_consensus.cc:374] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260927 23:28:02.424527 2378 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.60:41571, user_credentials={real_user=slave}} blocked reactor thread for 246998us
I20260927 23:28:02.428195 3311 raft_consensus.cc:482] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:02.436488 3311 raft_consensus.cc:504] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:02.438568 3311 leader_election.cc:288] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:02.458952 2384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_576d409caeb746d48f1b67bdc96437bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:02.460162 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 39b3bf40c51646d38559f6852ebad6f0 because the tablet was not found. No further retry: Not found: Tablet not found: 39b3bf40c51646d38559f6852ebad6f0
W20260927 23:28:02.461238 2384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_576d409caeb746d48f1b67bdc96437bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:02.462030 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 2572268ca47346b0b78abc6a9ca7f908 because the tablet was not found. No further retry: Not found: Tablet not found: 2572268ca47346b0b78abc6a9ca7f908
I20260927 23:28:02.462957 3298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:37745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/wal
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.57.62:38773,127.2.57.61:34987,127.2.57.60:41571
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.60:41571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.57.60
--webserver_port=39731
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
W20260927 23:28:02.463822 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 3c80915269094d3e8554c1655a1c98af because the tablet was not found. No further retry: Not found: Tablet not found: 3c80915269094d3e8554c1655a1c98af
I20260927 23:28:02.464938 3298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20260927 23:28:02.466773 2801 consensus_meta.cc:287] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51: Time spent flushing consensus metadata: real 0.602s user 0.007s sys 0.000s
I20260927 23:28:02.467587 3298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 23:28:02.483973 3368 raft_consensus.cc:482] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:02.493202 3312 raft_consensus.cc:482] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:02.493702 3312 raft_consensus.cc:504] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:02.495646 3312 leader_election.cc:288] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:02.485630 3368 raft_consensus.cc:504] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:02.506407 3368 leader_election.cc:288] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:02.507371 3342 tablet.cc:1722] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
W20260927 23:28:02.516631 3376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:02.523507 3167 tablet_bootstrap.cc:492] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
W20260927 23:28:02.528602 2945 consensus_meta.cc:287] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent flushing consensus metadata: real 0.718s user 0.009s sys 0.000s
I20260927 23:28:02.534297 3167 ts_tablet_manager.cc:1397] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.269s user 0.007s sys 0.006s
I20260927 23:28:02.536895 2391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet a80bff324902498fa5c04fe8cbc05f9d
I20260927 23:28:02.538539 2388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 45dd55b5a9d94ff0b84a799174ab5359
I20260927 23:28:02.540084 2388 catalog_manager.cc:6565] ReplaceTablet: tablet 45dd55b5a9d94ff0b84a799174ab5359 deleted and replaced by tablet 2d0d8a82c5234880ab8396b1777e36c8
I20260927 23:28:02.544059 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet 45dd55b5a9d94ff0b84a799174ab5359 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d0d8a82c5234880ab8396b1777e36c8) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:02.543800 2801 tablet_service.cc:1514] Processing DeleteTablet for tablet a80bff324902498fa5c04fe8cbc05f9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:02 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:02.544001 2389 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 43096d258a654532906f59d581f5a5d6 (table default.Table_3777b02a50e04654b10cc6bd6c73f51e [id=d3de7e8bfbed4409bf10ead542fe1c9d]). Expected version 2 got 0
I20260927 23:28:02.550450 2389 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 410294c28c49404e9c547b27111712eb (table default.Table_813e5d484f8f462b86e9136a57eebacd [id=b215370f7b594ab28c3e2a34de2a6f16]). Expected version 2 got 1
I20260927 23:28:02.551896 2389 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (table default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]). Expected version 1 got 0
I20260927 23:28:02.552604 2389 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]). Expected version 1 got 0
I20260927 23:28:02.554126 2389 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e (table default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2]). Expected version 1 got 0
I20260927 23:28:02.553964 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_f1571426f866470d94cfe827bd30ba79"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:02.555627 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f1571426f866470d94cfe827bd30ba79 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:02.555224 2386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1d299ca52cfc493499b5eaf0acdeb5f9 consensus state has no leader
I20260927 23:28:02.558845 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_b967cf863a6848919c6542cb6bbb1c70"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:02.561028 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b967cf863a6848919c6542cb6bbb1c70 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:02.562541 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_d2b79535214c487fb59255b089a1b5bd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:02.564391 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d2b79535214c487fb59255b089a1b5bd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:02.542943 3167 raft_consensus.cc:348] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:02.569422 3167 raft_consensus.cc:374] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:02.570549 3167 raft_consensus.cc:729] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:02.582361 3242 consensus_queue.cc:1035] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:02.579279 3167 consensus_queue.cc:260] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:02.596112 3336 tablet_replica.cc:330] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:02.589249 3341 ts_tablet_manager.cc:1918] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:02.597842 3341 log.cc:1196] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/3c80915269094d3e8554c1655a1c98af
I20260927 23:28:02.600461 3341 ts_tablet_manager.cc:1939] T 3c80915269094d3e8554c1655a1c98af P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:02.601938 3336 raft_consensus.cc:2229] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:02.602624 3336 raft_consensus.cc:2258] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:02.597249 3072 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.579s user 0.026s sys 0.010s
W20260927 23:28:02.605082 3072 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.587s user 0.026s sys 0.010s
I20260927 23:28:02.599071 3167 ts_tablet_manager.cc:1428] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.059s user 0.006s sys 0.001s
I20260927 23:28:02.607036 3072 tablet_bootstrap.cc:492] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:02.607524 3072 ts_tablet_manager.cc:1397] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.725s user 0.030s sys 0.018s
I20260927 23:28:02.612952 3167 tablet_bootstrap.cc:492] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:02.621695 3341 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 59) took 1027 ms. Trace:
I20260927 23:28:02.630908 3341 rpcz_store.cc:276] 0927 23:28:01.594755 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:01.662157 (+ 67402us) service_pool.cc:227] Handling call
0927 23:28:01.742491 (+ 80334us) ts_tablet_manager.cc:1145] Deleting tablet 3c80915269094d3e8554c1655a1c98af
0927 23:28:01.742555 (+ 64us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:02.441468 (+698913us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:02.588903 (+147435us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:02.621560 (+ 32657us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":35014,"thread_start_us":73130,"threads_started":1}
I20260927 23:28:02.635205 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 3c80915269094d3e8554c1655a1c98af (table default.Table_52009c39b67d47ea919822fecb9fcc5f [id=d4bab68cc19145baae5fab5691c88c8b]) successfully deleted
I20260927 23:28:02.641911 3341 tablet_replica.cc:330] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:02.647528 3341 raft_consensus.cc:2229] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:02.648234 3341 raft_consensus.cc:2258] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:02.650732 3341 ts_tablet_manager.cc:1905] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:02.644546 3167 tablet_bootstrap.cc:654] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:02.683934 3072 raft_consensus.cc:348] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:02.684686 3072 raft_consensus.cc:374] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:02.684942 3072 raft_consensus.cc:729] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:02.686297 3167 tablet_bootstrap.cc:492] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:02.686759 3167 ts_tablet_manager.cc:1397] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.075s user 0.009s sys 0.002s
I20260927 23:28:02.464890 3335 tablet_replica.cc:330] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:02.541733 2646 tablet_service.cc:1514] Processing DeleteTablet for tablet a80bff324902498fa5c04fe8cbc05f9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:02 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:02.541913 2652 tablet_service.cc:1514] Processing DeleteTablet for tablet 45dd55b5a9d94ff0b84a799174ab5359 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d0d8a82c5234880ab8396b1777e36c8) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:02.700951 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e104d49543934e5db3b188e1db065cdc" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:02.419548 2608 leader_election.cc:302] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:02.704531 3250 raft_consensus.cc:2781] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:02.705017 3250 raft_consensus.cc:686] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:02.705863 3250 consensus_queue.cc:237] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:28:02.704113 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 284632us
I20260927 23:28:02.716953 3336 ts_tablet_manager.cc:1905] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:02.716861 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e104d49543934e5db3b188e1db065cdc" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:02.723786 2609 consensus_peers.cc:487] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:02.725124 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80bff324902498fa5c04fe8cbc05f9d" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:02.733314 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d837daee7a04b338813519acf53dfae" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:02.733002 2385 catalog_manager.cc:5495] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe182b2afc64d908f692ef45eae1d51 (127.2.57.2). New cstate: current_term: 1 leader_uuid: "abe182b2afc64d908f692ef45eae1d51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: HEALTHY } } }
I20260927 23:28:02.641198 3374 tablet.cc:1722] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:02.736199 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]). Expected version 1 got 0
I20260927 23:28:02.732597 3072 consensus_queue.cc:260] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:02.736847 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 43096d258a654532906f59d581f5a5d6 (table default.Table_3777b02a50e04654b10cc6bd6c73f51e [id=d3de7e8bfbed4409bf10ead542fe1c9d]). Expected version 2 got 0
I20260927 23:28:02.737514 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 410294c28c49404e9c547b27111712eb (table default.Table_813e5d484f8f462b86e9136a57eebacd [id=b215370f7b594ab28c3e2a34de2a6f16]). Expected version 2 got 1
I20260927 23:28:02.738719 2390 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 3a4c099683d1426282efd580a552c97b
I20260927 23:28:02.739707 3335 raft_consensus.cc:2229] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:02.741883 3335 pending_rounds.cc:62] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1: Trying to abort 1 pending ops.
I20260927 23:28:02.742153 3335 pending_rounds.cc:68] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099685720797184 op_type: NO_OP noop_request { }
I20260927 23:28:02.742590 3335 raft_consensus.cc:2866] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:28:02.742954 3335 raft_consensus.cc:2258] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:02.744355 2390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_7c86d4308c504c4ba092bb6634326544" } modify_external_catalogs: true
I20260927 23:28:02.744918 2390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_7c86d4308c504c4ba092bb6634326544" } modify_external_catalogs: true
I20260927 23:28:02.750279 3335 ts_tablet_manager.cc:1905] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:02.758967 3167 raft_consensus.cc:348] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:02.759930 3167 raft_consensus.cc:374] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:02.763239 2388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_0cce1d1549d5401c8b6c9966fe2674f4" } new_table_name: "default.Table_9de2a295d1fc4fa692d084f283b81ba8" modify_external_catalogs: true
I20260927 23:28:02.766203 3167 raft_consensus.cc:729] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:02.767679 2393 master_service.cc:946] ReplaceTablet: received request to replace tablet 5efbd0211cf3408cb5ea9640d8da6694 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:02.769937 2652 tablet_service.cc:1514] Processing DeleteTablet for tablet 2572268ca47346b0b78abc6a9ca7f908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:02.770002 2389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 22) took 1092 ms (client timeout 1000 ms). Trace:
W20260927 23:28:02.770385 2389 rpcz_store.cc:269] 0927 23:28:01.677643 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.005591 (+327948us) service_pool.cc:227] Handling call
0927 23:28:02.769927 (+764336us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":17}
I20260927 23:28:02.773046 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 66f2842218fc42e5b1b83314e5f1a158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:02.773943 2646 tablet_service.cc:1514] Processing DeleteTablet for tablet 39b3bf40c51646d38559f6852ebad6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:02.774333 2392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 15) took 1298 ms (client timeout 1000 ms). Trace:
I20260927 23:28:02.775857 2652 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c80915269094d3e8554c1655a1c98af with delete_type TABLET_DATA_DELETED (replaced by tablet 3a4c099683d1426282efd580a552c97b) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:02.776799 3167 consensus_queue.cc:260] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:02.777405 2384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_33d39f037aca42d7a1e02a19788cf146" } new_table_name: "default.Table_29404653bfbb4e5d8192bcb604084df0" modify_external_catalogs: true
I20260927 23:28:02.417865 3070 raft_consensus.cc:729] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:02.782188 3070 consensus_queue.cc:260] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:02.794407 3167 ts_tablet_manager.cc:1428] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.107s user 0.006s sys 0.002s
I20260927 23:28:02.799226 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_585d1f792a1b42e7ae86d5ecb5edaff2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:02.801060 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_585d1f792a1b42e7ae86d5ecb5edaff2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:02.774678 2392 rpcz_store.cc:269] 0927 23:28:01.475873 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:01.980921 (+505048us) service_pool.cc:227] Handling call
0927 23:28:02.774270 (+793349us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:02.352894 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:02.353296 (+ 402us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:02.353321 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:28:02.353337 (+ 16us) tablet.cc:660] Decoding operations
0927 23:28:02.353664 (+ 327us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:02.353706 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:02.353730 (+ 24us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:02.354009 (+ 279us) tablet.cc:699] Row locks acquired
0927 23:28:02.354024 (+ 15us) write_op.cc:260] PREPARE: finished
0927 23:28:02.354147 (+ 123us) write_op.cc:270] Start()
0927 23:28:02.354245 (+ 98us) write_op.cc:276] Timestamp: P: 1790551682354106 usec, L: 0
0927 23:28:02.354266 (+ 21us) op_driver.cc:331] REPLICATION: starting
0927 23:28:02.354839 (+ 573us) log.cc:842] Serialized 895 byte log entry
0927 23:28:02.384880 (+ 30041us) op_driver.cc:444] REPLICATION: finished
0927 23:28:02.396234 (+ 11354us) write_op.cc:301] APPLY: starting
0927 23:28:02.396280 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:02.396419 (+ 139us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:02.396542 (+ 123us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:02.396781 (+ 239us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:02.396921 (+ 140us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:02.396967 (+ 46us) write_op.cc:312] APPLY: finished
0927 23:28:02.397174 (+ 207us) log.cc:842] Serialized 29 byte log entry
0927 23:28:02.397306 (+ 132us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:02.397378 (+ 72us) write_op.cc:454] Released schema lock
0927 23:28:02.397412 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:02.397584 (+ 172us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:02.397607 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":56007,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":11216,"apply.run_cpu_time_us":1550,"apply.run_wall_time_us":1551,"num_ops":2,"prepare.queue_time_us":4288,"prepare.run_cpu_time_us":2289,"prepare.run_wall_time_us":3677,"raft.queue_time_us":4449,"raft.run_cpu_time_us":9,"raft.run_wall_time_us":10,"replication_time_us":30544}]]}
W20260927 23:28:02.815289 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 84944.4us
W20260927 23:28:02.816005 2907 leader_election.cc:341] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:02.817073 2907 leader_election.cc:341] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:02.804500 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 3f71f948891b4d7fa95c3a1fbd92ce1e (DEFAULT_TABLE table=default.Table_047c5f89196f446e86279aae36cf7f99 [id=2aecf7c487414df1985aa5f284224515]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:02.820838 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f71f948891b4d7fa95c3a1fbd92ce1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:02.820922 3311 consensus_queue.cc:1035] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:02.822054 3311 raft_consensus.cc:482] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:02.822427 3311 raft_consensus.cc:504] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:02.823377 2944 tablet_service.cc:1514] Processing DeleteTablet for tablet 517d7bfc225a4372bfd79682a550a0d1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:02.825590 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d837daee7a04b338813519acf53dfae" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:02.827831 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80bff324902498fa5c04fe8cbc05f9d" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:02.829319 2907 leader_election.cc:341] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:02.829493 3312 consensus_queue.cc:1035] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:02.824061 3311 leader_election.cc:288] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:02.836478 3072 ts_tablet_manager.cc:1428] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.229s user 0.006s sys 0.000s
I20260927 23:28:02.837324 3072 tablet_bootstrap.cc:492] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:02.842278 3072 tablet_bootstrap.cc:654] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:02.856278 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 3f71f948891b4d7fa95c3a1fbd92ce1e (DEFAULT_TABLE table=default.Table_047c5f89196f446e86279aae36cf7f99 [id=2aecf7c487414df1985aa5f284224515]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:02.857388 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f71f948891b4d7fa95c3a1fbd92ce1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:02.862397 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "933d4c631a1345f9a117b87d0e669117" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:02.873270 3167 tablet_bootstrap.cc:492] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
W20260927 23:28:02.874181 2388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 04f670db808c4e289c655679da9d2d9d consensus state has no leader
I20260927 23:28:02.875628 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_353f3faa65ca4c0690336084abe3e912"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:02.877816 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_353f3faa65ca4c0690336084abe3e912 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:02.882658 2388 master_service.cc:946] ReplaceTablet: received request to replace tablet 9cce47b050b445dfba7be5ea3b6a8636 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:02.898888 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 43096d258a654532906f59d581f5a5d6 (table default.Table_3777b02a50e04654b10cc6bd6c73f51e [id=d3de7e8bfbed4409bf10ead542fe1c9d]). Expected version 2 got 0
I20260927 23:28:02.907569 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c80915269094d3e8554c1655a1c98af with delete_type TABLET_DATA_DELETED (replaced by tablet 3a4c099683d1426282efd580a552c97b) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:02.908576 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (table default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]). Expected version 1 got 0
I20260927 23:28:02.909060 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]). Expected version 1 got 0
I20260927 23:28:02.843060 2944 tablet_service.cc:1514] Processing DeleteTablet for tablet 39b3bf40c51646d38559f6852ebad6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:02.910022 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e (table default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2]). Expected version 1 got 0
I20260927 23:28:02.843807 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c80915269094d3e8554c1655a1c98af with delete_type TABLET_DATA_DELETED (replaced by tablet 3a4c099683d1426282efd580a552c97b) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:02.844512 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 828f870365244777b59e8a0756d7d4c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 5bf856a05dff4fb4943fe8e1d9b7f385) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:02.910492 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1d299ca52cfc493499b5eaf0acdeb5f9 (table default.Table_bf18dd7dd7984680bf41384772c15e61 [id=1af28dc40a8d408ba31c695e8563efc2]). Expected version 1 got 0
I20260927 23:28:02.845221 2945 tablet_service.cc:1514] Processing DeleteTablet for tablet 7096e434ac9f4290ad58ba4f7e7c2513 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:44952
W20260927 23:28:02.847208 2906 leader_election.cc:341] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:02.852591 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet a80bff324902498fa5c04fe8cbc05f9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:02 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:02.853565 2951 tablet_service.cc:1514] Processing DeleteTablet for tablet 45dd55b5a9d94ff0b84a799174ab5359 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d0d8a82c5234880ab8396b1777e36c8) from {username='slave'} at 127.0.0.1:44952
W20260927 23:28:02.899626 2907 leader_election.cc:341] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:02.911015 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 410294c28c49404e9c547b27111712eb (table default.Table_813e5d484f8f462b86e9136a57eebacd [id=b215370f7b594ab28c3e2a34de2a6f16]). Expected version 2 got 1
I20260927 23:28:02.911890 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet fbfa55a66b8042ed8592fbff31e92d55 (table default.Table_2b44b865509b414ba9c0c667dd764097 [id=3b845f646d7f4af99b7889442b4d7d0f]). Expected version 1 got 0
I20260927 23:28:02.912060 2906 leader_election.cc:302] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:02.912500 3335 ts_tablet_manager.cc:1918] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:02.912665 2390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 5efbd0211cf3408cb5ea9640d8da6694
I20260927 23:28:02.912945 3335 log.cc:1196] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/828f870365244777b59e8a0756d7d4c9
I20260927 23:28:02.913319 3311 raft_consensus.cc:2726] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:02.913679 3335 ts_tablet_manager.cc:1939] T 828f870365244777b59e8a0756d7d4c9 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:02.915222 2646 tablet_service.cc:1514] Processing DeleteTablet for tablet 5efbd0211cf3408cb5ea9640d8da6694 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:02 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:02.916899 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 828f870365244777b59e8a0756d7d4c9 (table default.Table_25f0e3650873407a8b3cbd8c8e0c86af [id=48f252a9cdb34eb1b452540f4186285c]) successfully deleted
I20260927 23:28:02.918499 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet 5efbd0211cf3408cb5ea9640d8da6694 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:02 UTC) from {username='slave'} at 127.0.0.1:42492
W20260927 23:28:02.918862 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 71681.5us
W20260927 23:28:02.922719 2906 leader_election.cc:341] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:02.923378 3070 ts_tablet_manager.cc:1428] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.538s user 0.008s sys 0.002s
W20260927 23:28:02.923784 3070 ts_tablet_manager.cc:1460] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: Tablet startup took 538ms
I20260927 23:28:02.924898 2906 leader_election.cc:302] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:02.924085 3070 ts_tablet_manager.cc:1462] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: Trace:
0927 23:27:59.224103 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:27:59.224238 (+ 135us) service_pool.cc:227] Handling call
0927 23:27:59.225728 (+ 1490us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:27:59.225828 (+ 100us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:27:59.295562 (+ 69734us) tablet_metadata.cc:641] Metadata flushed
0927 23:27:59.546877 (+251315us) tablet_replica.cc:205] Creating consensus instance
0927 23:27:59.547413 (+ 536us) inbound_call.cc:173] Queueing success response
0927 23:28:02.244793 (+2697380us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0927 23:28:02.385206 (+140413us) ts_tablet_manager.cc:1429] Starting tablet replica
0927 23:28:02.385803 (+ 597us) tablet_replica.cc:259] Starting instrumentation
0927 23:28:02.386342 (+ 539us) tablet_replica.cc:278] Starting consensus
0927 23:28:02.871705 (+485363us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b5400150aa8. stack: 00007fe1d7372033 00007fe1d7371f30 00007fe1d6ef3534 00000000004db2b4 00000000004db249 00007fe1e0795e91 00007fe1e0794afc 00007fe1de712ff7 00007fe1de71c426 00007fe1df3234fa 00007fe1df3372d1 00007fe1df337279 00007fe1df337209 00007fe1df3371d1 00007fe1df3364ad 0000000000580784
Metrics: {"raft.queue_time_us":4204,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":190,"spinlock_wait_cycles":51858688,"tablet-open.queue_time_us":2697435}
I20260927 23:28:02.926750 3164 raft_consensus.cc:2726] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:02.926819 3070 tablet_bootstrap.cc:492] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
W20260927 23:28:02.927105 2906 leader_election.cc:341] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:02.927505 2906 leader_election.cc:302] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:02.927816 3385 log.cc:924] Time spent T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d: Append to log took a long time: real 0.080s user 0.000s sys 0.000s
I20260927 23:28:02.928429 3164 raft_consensus.cc:2726] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:02.928576 2906 consensus_peers.cc:487] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:02.908221 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_f298fb86e27d466eaee4ae90137687d0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:02.937737 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f298fb86e27d466eaee4ae90137687d0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:02.939677 3374 tablet.cc:1722] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:02.946727 3167 tablet_bootstrap.cc:654] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:02.952319 3341 ts_tablet_manager.cc:1918] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:02.952878 3341 log.cc:1196] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/39b3bf40c51646d38559f6852ebad6f0
I20260927 23:28:02.954005 3341 ts_tablet_manager.cc:1939] T 39b3bf40c51646d38559f6852ebad6f0 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:02.958204 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 39b3bf40c51646d38559f6852ebad6f0 (table default.Table_faba7f79481947c1a44139eddeb749a6 [id=2aa2af3d524a455e8d22c0171fe10942]) successfully deleted
I20260927 23:28:02.959785 3341 tablet_replica.cc:330] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:02.960772 3341 raft_consensus.cc:2229] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:02.961594 3341 raft_consensus.cc:2258] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:02.965826 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe (DEFAULT_TABLE table=default.Table_e57880e3ffa64d3b876813b8c2246002 [id=805a37d784f64efe9c46c985e5c853f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:02.966396 3167 tablet_bootstrap.cc:492] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:02.966843 3167 ts_tablet_manager.cc:1397] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.094s user 0.005s sys 0.008s
I20260927 23:28:02.967162 3341 ts_tablet_manager.cc:1905] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:02.967338 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:02.969231 3167 raft_consensus.cc:348] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:02.969916 3167 raft_consensus.cc:374] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:02.970149 3167 raft_consensus.cc:729] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:02.985033 3167 consensus_queue.cc:260] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:02.986403 2806 tablet_service.cc:1467] Processing CreateTablet for tablet ac19099dd7bf4b669d504a654ee69482 (DEFAULT_TABLE table=default.Table_1cf287cdda1046558956f74f784c2536 [id=8beb1e4757014f69ba48b5ea31b27afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:02.987434 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac19099dd7bf4b669d504a654ee69482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:02.988998 2803 tablet_service.cc:1467] Processing CreateTablet for tablet cfbbe17f54d1418b95efc3943db0b6c1 (DEFAULT_TABLE table=default.Table_2ec8e4445f1d425ea0a7dae9f9d6200b [id=f21b53d8e90a48bc9cb7f6268c946cc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:02.989959 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfbbe17f54d1418b95efc3943db0b6c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:02.994186 3167 ts_tablet_manager.cc:1428] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.027s user 0.006s sys 0.001s
I20260927 23:28:02.995326 3167 tablet_bootstrap.cc:492] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:03.026036 3341 ts_tablet_manager.cc:1918] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260927 23:28:03.029106 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet a80bff324902498fa5c04fe8cbc05f9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a80bff324902498fa5c04fe8cbc05f9d already in progress: creating tablet
I20260927 23:28:03.032581 3341 log.cc:1196] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/2572268ca47346b0b78abc6a9ca7f908
I20260927 23:28:03.033598 3341 ts_tablet_manager.cc:1939] T 2572268ca47346b0b78abc6a9ca7f908 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:03.033530 2392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_877402b3ef234b9c99bbdab0842b3885" } modify_external_catalogs: true
I20260927 23:28:03.034183 2392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_877402b3ef234b9c99bbdab0842b3885" } modify_external_catalogs: true
I20260927 23:28:03.037057 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 2572268ca47346b0b78abc6a9ca7f908 (table default.Table_2d889ed8e18e47a79f0541db93242386 [id=103d7c5e210c48fdafb8c199eb4cf676]) successfully deleted
I20260927 23:28:03.038100 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_3f55d539f6264cef8baea2cc68a8fe37"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:03.040040 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3f55d539f6264cef8baea2cc68a8fe37 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:03.053145 3167 tablet_bootstrap.cc:654] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:03.056800 3341 tablet_replica.cc:330] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:03.057801 3341 raft_consensus.cc:2229] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:03.058336 3341 raft_consensus.cc:2258] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:03.060588 3341 ts_tablet_manager.cc:1905] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:03.066341 3070 tablet_bootstrap.cc:700] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent opening tablet: real 0.122s user 0.003s sys 0.000s
I20260927 23:28:03.066996 3070 tablet_bootstrap.cc:654] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:03.068001 3112 raft_consensus.cc:482] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:03.068447 3112 raft_consensus.cc:504] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.069869 3167 tablet_bootstrap.cc:492] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:03.081863 2388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 9cce47b050b445dfba7be5ea3b6a8636
I20260927 23:28:03.083336 2388 catalog_manager.cc:6565] ReplaceTablet: tablet 9cce47b050b445dfba7be5ea3b6a8636 deleted and replaced by tablet 1da38499af164d7fb5c1a8d65167ad24
I20260927 23:28:03.085469 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe (DEFAULT_TABLE table=default.Table_e57880e3ffa64d3b876813b8c2246002 [id=805a37d784f64efe9c46c985e5c853f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:03.086714 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:03.089058 2648 tablet_service.cc:1467] Processing CreateTablet for tablet cfbbe17f54d1418b95efc3943db0b6c1 (DEFAULT_TABLE table=default.Table_2ec8e4445f1d425ea0a7dae9f9d6200b [id=f21b53d8e90a48bc9cb7f6268c946cc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:03.090144 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfbbe17f54d1418b95efc3943db0b6c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:03.095984 2801 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cce47b050b445dfba7be5ea3b6a8636 with delete_type TABLET_DATA_DELETED (replaced by tablet 1da38499af164d7fb5c1a8d65167ad24) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:03.104830 3335 tablet_replica.cc:330] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:03.106097 2653 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cce47b050b445dfba7be5ea3b6a8636 with delete_type TABLET_DATA_DELETED (replaced by tablet 1da38499af164d7fb5c1a8d65167ad24) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:03.106995 2647 tablet_service.cc:1467] Processing CreateTablet for tablet ac19099dd7bf4b669d504a654ee69482 (DEFAULT_TABLE table=default.Table_1cf287cdda1046558956f74f784c2536 [id=8beb1e4757014f69ba48b5ea31b27afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:03.109467 3167 ts_tablet_manager.cc:1397] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.077s user 0.007s sys 0.005s
I20260927 23:28:03.118198 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac19099dd7bf4b669d504a654ee69482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:03.132036 3167 raft_consensus.cc:348] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.135918 3167 raft_consensus.cc:374] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:03.140604 3167 raft_consensus.cc:729] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:03.142405 3167 consensus_queue.cc:260] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.149001 2388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_353f3faa65ca4c0690336084abe3e912" } new_table_name: "default.Table_58948ac6bf854c31a26e8a6d47456c90" modify_external_catalogs: true
I20260927 23:28:03.149991 3335 raft_consensus.cc:2229] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:03.150969 3335 raft_consensus.cc:2258] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:03.152514 2386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_9de2a295d1fc4fa692d084f283b81ba8" } modify_external_catalogs: true
I20260927 23:28:03.153038 2386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_9de2a295d1fc4fa692d084f283b81ba8" } modify_external_catalogs: true
I20260927 23:28:03.158624 3335 ts_tablet_manager.cc:1905] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:03.161118 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 232473us
W20260927 23:28:03.167500 2906 consensus_peers.cc:487] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:03.167711 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "933d4c631a1345f9a117b87d0e669117" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:03.175395 2906 leader_election.cc:341] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:03.175966 2906 leader_election.cc:302] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:03.178733 3392 consensus_queue.cc:1035] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:28:03.183101 2380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 04f670db808c4e289c655679da9d2d9d (table default.Table_9de2a295d1fc4fa692d084f283b81ba8 [id=bd8e999821b44f628e5ec3b1d7915773], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 04f670db808c4e289c655679da9d2d9d consensus state has no leader
I20260927 23:28:03.183723 3167 ts_tablet_manager.cc:1428] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.072s user 0.003s sys 0.002s
I20260927 23:28:03.198962 3112 leader_election.cc:288] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:03.201108 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5efbd0211cf3408cb5ea9640d8da6694" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:03.202984 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet 66f2842218fc42e5b1b83314e5f1a158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:03.203917 2944 tablet_service.cc:1467] Processing CreateTablet for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe (DEFAULT_TABLE table=default.Table_e57880e3ffa64d3b876813b8c2246002 [id=805a37d784f64efe9c46c985e5c853f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:03.205260 2950 tablet_service.cc:1467] Processing CreateTablet for tablet cfbbe17f54d1418b95efc3943db0b6c1 (DEFAULT_TABLE table=default.Table_2ec8e4445f1d425ea0a7dae9f9d6200b [id=f21b53d8e90a48bc9cb7f6268c946cc1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:03.206099 2945 tablet_service.cc:1514] Processing DeleteTablet for tablet 5efbd0211cf3408cb5ea9640d8da6694 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:02 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:03.206758 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfbbe17f54d1418b95efc3943db0b6c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:03.207828 2946 tablet_service.cc:1467] Processing CreateTablet for tablet ac19099dd7bf4b669d504a654ee69482 (DEFAULT_TABLE table=default.Table_1cf287cdda1046558956f74f784c2536 [id=8beb1e4757014f69ba48b5ea31b27afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:03.209302 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac19099dd7bf4b669d504a654ee69482. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:03.211707 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 3f71f948891b4d7fa95c3a1fbd92ce1e (DEFAULT_TABLE table=default.Table_047c5f89196f446e86279aae36cf7f99 [id=2aecf7c487414df1985aa5f284224515]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:03.212992 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f71f948891b4d7fa95c3a1fbd92ce1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:03.216900 2949 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cce47b050b445dfba7be5ea3b6a8636 with delete_type TABLET_DATA_DELETED (replaced by tablet 1da38499af164d7fb5c1a8d65167ad24) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:03.217324 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5efbd0211cf3408cb5ea9640d8da6694" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:03.218485 2968 raft_consensus.cc:2454] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:03.184803 3167 tablet_bootstrap.cc:492] T fbfa55a66b8042ed8592fbff31e92d55 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:03.233606 2608 leader_election.cc:302] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:02.957346 3399 tablet.cc:1722] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:03.234762 3112 raft_consensus.cc:2781] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:03.235221 3112 raft_consensus.cc:482] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:03.235574 3112 raft_consensus.cc:3037] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:03.236701 2609 leader_election.cc:341] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:03.243588 2390 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]). Expected version 1 got 0
I20260927 23:28:03.244261 2390 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 43096d258a654532906f59d581f5a5d6 (table default.Table_3777b02a50e04654b10cc6bd6c73f51e [id=d3de7e8bfbed4409bf10ead542fe1c9d]). Expected version 2 got 0
I20260927 23:28:03.246279 2390 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e (table default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2]). Expected version 1 got 0
I20260927 23:28:03.247074 3392 raft_consensus.cc:482] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:03.247579 3392 raft_consensus.cc:504] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.250667 3392 leader_election.cc:288] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:03.253275 2763 leader_election.cc:341] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 7096e434ac9f4290ad58ba4f7e7c2513
W20260927 23:28:03.259123 2388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 7f530db3b52944f6a283ee9e2bf66828 has no consensus state
I20260927 23:28:03.263644 2393 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 410294c28c49404e9c547b27111712eb (table default.Table_813e5d484f8f462b86e9136a57eebacd [id=b215370f7b594ab28c3e2a34de2a6f16]). Expected version 2 got 1
I20260927 23:28:03.264760 2393 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_d8d7e47a77aa46818639a40ebd279a12 [id=29d80c3f3f21428b819d211d3e10f621]). Expected version 1 got 0
I20260927 23:28:03.264995 3407 tablet.cc:1722] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:03.265324 2393 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:03.266002 2393 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 43096d258a654532906f59d581f5a5d6 (table default.Table_3777b02a50e04654b10cc6bd6c73f51e [id=d3de7e8bfbed4409bf10ead542fe1c9d]). Expected version 2 got 0
I20260927 23:28:03.277814 3335 ts_tablet_manager.cc:1918] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:03.282796 3335 log.cc:1196] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/45dd55b5a9d94ff0b84a799174ab5359
I20260927 23:28:03.292515 3072 tablet_bootstrap.cc:492] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:03.293000 3072 ts_tablet_manager.cc:1397] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.456s user 0.011s sys 0.000s
I20260927 23:28:03.300676 3368 raft_consensus.cc:482] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:03.301141 3368 raft_consensus.cc:504] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.303247 3368 leader_election.cc:288] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:03.303669 3368 raft_consensus.cc:482] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:03.306991 3335 ts_tablet_manager.cc:1939] T 45dd55b5a9d94ff0b84a799174ab5359 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:03.308768 3311 raft_consensus.cc:2726] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:03.316745 3335 tablet_replica.cc:330] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:03.317935 3335 raft_consensus.cc:2229] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:03.320348 3335 raft_consensus.cc:2258] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:03.320505 3167 tablet_bootstrap.cc:654] T fbfa55a66b8042ed8592fbff31e92d55 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:03.314759 2944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:03.328550 3070 tablet_bootstrap.cc:492] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:03.329016 3070 ts_tablet_manager.cc:1397] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.402s user 0.015s sys 0.004s
I20260927 23:28:03.329924 3335 ts_tablet_manager.cc:1905] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:03.331261 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95216fdc2e104a2e9fc4bc424bdbe5e9" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:03.331722 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95216fdc2e104a2e9fc4bc424bdbe5e9" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:03.332408 2824 raft_consensus.cc:2454] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:03.332785 2672 raft_consensus.cc:2454] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:03.337091 3405 tablet.cc:1722] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:03.346009 2388 master_service.cc:946] ReplaceTablet: received request to replace tablet 3acdfe7faa1a4d5d8e340f09f1cae93e from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:03.348321 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 45dd55b5a9d94ff0b84a799174ab5359 (table default.Table_72f38013370a483784dfabf8e4646d24 [id=c7a02939c757489cb0b7318f5ed346ec]) successfully deleted
W20260927 23:28:03.349565 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 2572268ca47346b0b78abc6a9ca7f908 because the tablet was not found. No further retry: Not found: Tablet not found: 2572268ca47346b0b78abc6a9ca7f908
I20260927 23:28:03.356499 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_decc7c294d064d7daea036b101d04d55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:03.357926 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_decc7c294d064d7daea036b101d04d55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:03.360754 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_92117e5358444ef7bb1d38724fd8f933"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:03.362102 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_92117e5358444ef7bb1d38724fd8f933 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:03.373852 3070 raft_consensus.cc:348] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.374639 3070 raft_consensus.cc:374] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:03.374994 3070 raft_consensus.cc:729] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:03.375633 3070 consensus_queue.cc:260] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.347064 2393 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet fbfa55a66b8042ed8592fbff31e92d55 (table default.Table_2b44b865509b414ba9c0c667dd764097 [id=3b845f646d7f4af99b7889442b4d7d0f]). Expected version 1 got 0
I20260927 23:28:03.380666 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_49c3f2d8a1b14c02878df5f1934bbaba"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:03.382815 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_49c3f2d8a1b14c02878df5f1934bbaba in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:03.388073 2393 catalog_manager.cc:5495] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:03.369120 2906 leader_election.cc:302] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:03.323794 3312 raft_consensus.cc:482] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:03.304013 3368 raft_consensus.cc:504] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:03.425622 3070 ts_tablet_manager.cc:1428] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.096s user 0.006s sys 0.000s
I20260927 23:28:03.427989 3335 ts_tablet_manager.cc:1918] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:03.428540 3335 log.cc:1196] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/66f2842218fc42e5b1b83314e5f1a158
I20260927 23:28:03.429459 3335 ts_tablet_manager.cc:1939] T 66f2842218fc42e5b1b83314e5f1a158 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:03.437737 3072 raft_consensus.cc:348] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:03.438436 3072 raft_consensus.cc:374] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:03.438715 3072 raft_consensus.cc:729] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:03.439239 3072 consensus_queue.cc:260] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:03.442516 3415 tablet.cc:1722] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:03.421257 3312 raft_consensus.cc:504] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:28:03.444651 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 52263.3us
I20260927 23:28:03.446059 3312 leader_election.cc:288] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:03.447193 3341 ts_tablet_manager.cc:1918] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:03.449072 3335 tablet_replica.cc:330] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:03.444056 3311 raft_consensus.cc:2781] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 23:28:03.455150 3311 raft_consensus.cc:482] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:03.455470 3311 raft_consensus.cc:3037] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Advancing to term 2
W20260927 23:28:03.442004 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 73004.2us
I20260927 23:28:03.430843 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 66f2842218fc42e5b1b83314e5f1a158 (table default.Table_33057639607344a0bcd2f4850a382324 [id=6b1d660c32a74492bb7b47fea2e1132d]) successfully deleted
W20260927 23:28:03.457858 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 39b3bf40c51646d38559f6852ebad6f0 because the tablet was not found. No further retry: Not found: Tablet not found: 39b3bf40c51646d38559f6852ebad6f0
W20260927 23:28:03.458477 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 3c80915269094d3e8554c1655a1c98af because the tablet was not found. No further retry: Not found: Tablet not found: 3c80915269094d3e8554c1655a1c98af
I20260927 23:28:03.434151 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_91684f2e139444c0a8175fb9d1998733"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:03.460489 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_91684f2e139444c0a8175fb9d1998733 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:03.463419 2389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_746dd85f958f4346bb7c3484f59e75f4" } new_table_name: "default.Table_92a19ba603ef488c972786364cf49c70" modify_external_catalogs: true
W20260927 23:28:03.467164 2907 leader_election.cc:341] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 7096e434ac9f4290ad58ba4f7e7c2513
I20260927 23:28:03.451361 3405 tablet.cc:1722] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:03.470119 3341 log.cc:1196] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/828f870365244777b59e8a0756d7d4c9
I20260927 23:28:03.470886 3341 ts_tablet_manager.cc:1939] T 828f870365244777b59e8a0756d7d4c9 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:03.473109 3341 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 70) took 1172 ms. Trace:
I20260927 23:28:03.390748 3112 raft_consensus.cc:504] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.478751 2970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5efbd0211cf3408cb5ea9640d8da6694" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:03.479362 2970 raft_consensus.cc:3037] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:03.489895 3112 leader_election.cc:288] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:03.492565 3072 ts_tablet_manager.cc:1428] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.199s user 0.004s sys 0.003s
I20260927 23:28:03.493587 3072 tablet_bootstrap.cc:492] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
W20260927 23:28:03.505056 2378 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.60:41571, user_credentials={real_user=slave}} blocked reactor thread for 205386us
I20260927 23:28:03.505220 3341 rpcz_store.cc:276] 0927 23:28:02.300398 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.304561 (+ 4163us) service_pool.cc:227] Handling call
0927 23:28:03.056605 (+752044us) ts_tablet_manager.cc:1145] Deleting tablet 828f870365244777b59e8a0756d7d4c9
0927 23:28:03.056660 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:03.318900 (+262240us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:03.446927 (+128027us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:03.473018 (+ 26091us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":747261}
I20260927 23:28:03.506819 3341 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 69) took 1206 ms. Trace:
I20260927 23:28:03.506795 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 828f870365244777b59e8a0756d7d4c9 (table default.Table_25f0e3650873407a8b3cbd8c8e0c86af [id=48f252a9cdb34eb1b452540f4186285c]) successfully deleted
I20260927 23:28:03.506556 3420 tablet.cc:1722] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:03.507143 3341 rpcz_store.cc:276] 0927 23:28:02.300059 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.302513 (+ 2454us) service_pool.cc:227] Handling call
0927 23:28:03.506492 (+1203979us) ts_tablet_manager.cc:1145] Deleting tablet 517d7bfc225a4372bfd79682a550a0d1
0927 23:28:03.506540 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:03.506750 (+ 210us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1195291}
I20260927 23:28:03.508361 3341 tablet_replica.cc:330] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
W20260927 23:28:03.508801 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 517d7bfc225a4372bfd79682a550a0d1 because the tablet was not found. No further retry: Not found: Tablet not found: 517d7bfc225a4372bfd79682a550a0d1
I20260927 23:28:03.509284 3341 raft_consensus.cc:2229] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:03.509836 3341 raft_consensus.cc:2258] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:03.510823 2393 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e (table default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2]). Expected version 1 got 0
I20260927 23:28:03.511575 3335 raft_consensus.cc:2229] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:03.513296 2393 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 1d299ca52cfc493499b5eaf0acdeb5f9 (table default.Table_bf18dd7dd7984680bf41384772c15e61 [id=1af28dc40a8d408ba31c695e8563efc2]). Expected version 1 got 0
I20260927 23:28:03.515544 2393 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
I20260927 23:28:03.466259 3167 tablet_bootstrap.cc:492] T fbfa55a66b8042ed8592fbff31e92d55 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:03.476199 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5efbd0211cf3408cb5ea9640d8da6694" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:03.516674 2384 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cc1c0da6ea5c4a09a71555f37becae33 consensus state has no leader
I20260927 23:28:03.459296 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7096e434ac9f4290ad58ba4f7e7c2513" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:03.517596 2821 raft_consensus.cc:2454] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:03.518275 2392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 95216fdc2e104a2e9fc4bc424bdbe5e9
W20260927 23:28:03.519194 2609 leader_election.cc:341] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:03.520000 3167 ts_tablet_manager.cc:1397] T fbfa55a66b8042ed8592fbff31e92d55 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.335s user 0.017s sys 0.004s
I20260927 23:28:03.520787 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:03.522437 3167 raft_consensus.cc:348] T fbfa55a66b8042ed8592fbff31e92d55 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.523092 3167 raft_consensus.cc:374] T fbfa55a66b8042ed8592fbff31e92d55 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:03.523332 3167 raft_consensus.cc:729] T fbfa55a66b8042ed8592fbff31e92d55 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
W20260927 23:28:03.527815 2391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 23) took 828 ms (client timeout 1000 ms). Trace:
I20260927 23:28:03.522243 2390 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_813e5d484f8f462b86e9136a57eebacd [id=b215370f7b594ab28c3e2a34de2a6f16] alter complete (version 2)
I20260927 23:28:03.532193 2906 leader_election.cc:302] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1
I20260927 23:28:03.535022 3370 raft_consensus.cc:2781] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:03.535377 3370 raft_consensus.cc:482] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:03.535682 3370 raft_consensus.cc:3037] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:03.545835 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884b80a1d31349c98f75525416b3c527" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:03.547124 2906 leader_election.cc:341] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:03.553354 3167 consensus_queue.cc:260] T fbfa55a66b8042ed8592fbff31e92d55 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.556041 3370 raft_consensus.cc:504] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.560003 3167 ts_tablet_manager.cc:1428] T fbfa55a66b8042ed8592fbff31e92d55 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.040s user 0.005s sys 0.001s
I20260927 23:28:03.569623 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884b80a1d31349c98f75525416b3c527" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:03.571041 2907 leader_election.cc:341] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:03.571445 2907 leader_election.cc:302] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:03.572263 3164 raft_consensus.cc:2726] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:03.572975 3368 leader_election.cc:288] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:03.574664 3311 raft_consensus.cc:504] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.578195 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95216fdc2e104a2e9fc4bc424bdbe5e9" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:03.578764 2673 raft_consensus.cc:3037] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Advancing to term 2
I20260927 23:28:03.561192 3167 tablet_bootstrap.cc:492] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:03.583348 3397 raft_consensus.cc:482] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:03.583796 3397 raft_consensus.cc:504] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.585848 3397 leader_election.cc:288] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:03.586237 2673 raft_consensus.cc:2454] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 2.
I20260927 23:28:03.587021 3167 tablet_bootstrap.cc:654] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:03.587275 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cce47b050b445dfba7be5ea3b6a8636" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:03.571728 3341 ts_tablet_manager.cc:1905] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:03.588804 2609 leader_election.cc:341] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:03.590544 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cce47b050b445dfba7be5ea3b6a8636" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:03.591109 2972 raft_consensus.cc:2454] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:03.592208 2608 leader_election.cc:302] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:03.593371 2907 leader_election.cc:302] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:03.594344 3397 raft_consensus.cc:2781] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:03.594684 3397 raft_consensus.cc:482] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:03.594978 3397 raft_consensus.cc:3037] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:03.596053 3368 raft_consensus.cc:2781] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 2 FOLLOWER]: Leader election won for term 2
I20260927 23:28:03.596972 3368 raft_consensus.cc:686] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 2 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:03.597903 3368 consensus_queue.cc:237] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.600932 2805 tablet_service.cc:1514] Processing DeleteTablet for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:03.602442 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet 39b3bf40c51646d38559f6852ebad6f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:03.604694 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet 2572268ca47346b0b78abc6a9ca7f908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:03.608035 3072 tablet_bootstrap.cc:654] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:03.608589 2384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_21343cdd532b49f4ad94e82d0187b8d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:03.611090 2384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_21343cdd532b49f4ad94e82d0187b8d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:03.611676 3336 ts_tablet_manager.cc:1918] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:03.612128 3336 log.cc:1196] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/2572268ca47346b0b78abc6a9ca7f908
I20260927 23:28:03.616218 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_3bae57755da641d491998ef6ab5dd4ec"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:03.617697 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_3bae57755da641d491998ef6ab5dd4ec in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:03.600020 2970 raft_consensus.cc:2454] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
W20260927 23:28:03.622584 2389 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1da38499af164d7fb5c1a8d65167ad24 has no consensus state
I20260927 23:28:03.626732 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 45dd55b5a9d94ff0b84a799174ab5359 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d0d8a82c5234880ab8396b1777e36c8) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:03.627351 2646 tablet_service.cc:1514] Processing DeleteTablet for tablet 66f2842218fc42e5b1b83314e5f1a158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:03.628476 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 828f870365244777b59e8a0756d7d4c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 5bf856a05dff4fb4943fe8e1d9b7f385) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:03.634238 2390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_5924c026ff234c36b073a7fce24b05e6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:03.637527 3070 tablet_bootstrap.cc:492] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:03.638327 3397 raft_consensus.cc:504] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.641860 3167 tablet_bootstrap.cc:492] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:03.643447 3167 ts_tablet_manager.cc:1397] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.082s user 0.008s sys 0.003s
I20260927 23:28:03.645320 3336 ts_tablet_manager.cc:1939] T 2572268ca47346b0b78abc6a9ca7f908 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:03.645576 3167 raft_consensus.cc:348] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.646209 3167 raft_consensus.cc:374] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:03.646440 3167 raft_consensus.cc:729] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:03.647749 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 68) took 1609 ms. Trace:
I20260927 23:28:03.644024 3070 tablet_bootstrap.cc:654] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:03.651127 3415 tablet.cc:1722] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260927 23:28:03.636216 2390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5924c026ff234c36b073a7fce24b05e6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:03.652454 2907 leader_election.cc:341] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 7096e434ac9f4290ad58ba4f7e7c2513
I20260927 23:28:03.652842 3311 leader_election.cc:288] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:28:03.528144 2391 rpcz_store.cc:269] 0927 23:28:02.699857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.699979 (+ 122us) service_pool.cc:227] Handling call
0927 23:28:02.704535 (+ 4556us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:03.527759 (+823224us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8347,"reported_tablets":15,"spinlock_wait_cycles":3951360}
I20260927 23:28:03.660748 3341 ts_tablet_manager.cc:1918] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:03.661854 2392 master_service.cc:946] ReplaceTablet: received request to replace tablet 7767a32c07454fe6a92154429a3b6899 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:03.662155 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cce47b050b445dfba7be5ea3b6a8636" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:03.665464 3341 log.cc:1196] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/45dd55b5a9d94ff0b84a799174ab5359
I20260927 23:28:03.666621 3341 ts_tablet_manager.cc:1939] T 45dd55b5a9d94ff0b84a799174ab5359 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:03.670604 2608 leader_election.cc:302] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:03.671728 2609 leader_election.cc:341] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:03.674788 3341 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 72) took 1131 ms. Trace:
I20260927 23:28:03.675129 3341 rpcz_store.cc:276] 0927 23:28:02.543808 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.543926 (+ 118us) service_pool.cc:227] Handling call
0927 23:28:03.508167 (+964241us) ts_tablet_manager.cc:1145] Deleting tablet 45dd55b5a9d94ff0b84a799174ab5359
0927 23:28:03.508210 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:03.609671 (+101461us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:03.660519 (+ 50848us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:03.674719 (+ 14200us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":963625}
I20260927 23:28:03.662209 3167 consensus_queue.cc:260] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.677235 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 45dd55b5a9d94ff0b84a799174ab5359 (table default.Table_72f38013370a483784dfabf8e4646d24 [id=c7a02939c757489cb0b7318f5ed346ec]) successfully deleted
I20260927 23:28:03.680231 3167 ts_tablet_manager.cc:1428] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.037s user 0.004s sys 0.000s
I20260927 23:28:03.683769 3167 tablet_bootstrap.cc:492] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:03.685323 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_9c487cc2fe344578a7b995417de4aeac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:03.687048 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9c487cc2fe344578a7b995417de4aeac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:03.696768 3341 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 71) took 1153 ms. Trace:
I20260927 23:28:03.703348 3397 leader_election.cc:288] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:03.703079 3341 rpcz_store.cc:276] 0927 23:28:02.543463 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.543618 (+ 155us) service_pool.cc:227] Handling call
0927 23:28:03.696370 (+1152752us) ts_tablet_manager.cc:1145] Deleting tablet a80bff324902498fa5c04fe8cbc05f9d
0927 23:28:03.696420 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:03.696711 (+ 291us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1148501}
I20260927 23:28:03.705552 3341 tablet_replica.cc:330] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
W20260927 23:28:03.711277 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet a80bff324902498fa5c04fe8cbc05f9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a80bff324902498fa5c04fe8cbc05f9d already in progress: creating tablet
W20260927 23:28:03.712045 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 3c80915269094d3e8554c1655a1c98af because the tablet was not found. No further retry: Not found: Tablet not found: 3c80915269094d3e8554c1655a1c98af
W20260927 23:28:03.712814 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 5efbd0211cf3408cb5ea9640d8da6694 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5efbd0211cf3408cb5ea9640d8da6694 already in progress: creating tablet
I20260927 23:28:03.707540 3167 tablet_bootstrap.cc:654] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:03.708814 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95216fdc2e104a2e9fc4bc424bdbe5e9" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:03.713431 2821 raft_consensus.cc:3037] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 2
W20260927 23:28:03.713634 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 9cce47b050b445dfba7be5ea3b6a8636 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9cce47b050b445dfba7be5ea3b6a8636 already in progress: creating tablet
I20260927 23:28:03.648061 3336 rpcz_store.cc:276] 0927 23:28:02.038621 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.043196 (+ 4575us) service_pool.cc:227] Handling call
0927 23:28:02.413290 (+370094us) ts_tablet_manager.cc:1145] Deleting tablet 2572268ca47346b0b78abc6a9ca7f908
0927 23:28:02.584109 (+170819us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:03.024554 (+440445us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:03.505270 (+480716us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:03.647672 (+142402us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":352191}
I20260927 23:28:03.709440 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7096e434ac9f4290ad58ba4f7e7c2513" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:03.716498 2824 raft_consensus.cc:3037] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:03.716660 3336 tablet_replica.cc:330] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:03.722218 3336 raft_consensus.cc:2229] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:03.724720 2821 raft_consensus.cc:2454] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 2.
I20260927 23:28:03.727676 3336 pending_rounds.cc:62] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d: Trying to abort 1 pending ops.
I20260927 23:28:03.729009 3336 pending_rounds.cc:68] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099685720797184 op_type: NO_OP noop_request { }
I20260927 23:28:03.729417 3336 raft_consensus.cc:2866] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:28:03.728153 2824 raft_consensus.cc:2454] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:03.733695 2906 leader_election.cc:302] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1
I20260927 23:28:03.729821 3336 raft_consensus.cc:2258] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:03.736042 3311 raft_consensus.cc:2781] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:03.736503 3311 raft_consensus.cc:686] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:03.737099 3341 raft_consensus.cc:2229] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:03.737556 3341 raft_consensus.cc:2258] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:03.737226 3311 consensus_queue.cc:237] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.739271 3341 ts_tablet_manager.cc:1905] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:03.741343 3335 raft_consensus.cc:2258] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:03.741830 3250 raft_consensus.cc:2684] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Received election callback for term 1 while not running: Illegal state: RaftConsensus is not running: State = Stopped
I20260927 23:28:03.743568 3250 consensus_queue.cc:1035] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:03.724743 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cce47b050b445dfba7be5ea3b6a8636" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:03.753077 2969 raft_consensus.cc:3037] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:03.754886 3335 ts_tablet_manager.cc:1905] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:03.764803 2392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 7767a32c07454fe6a92154429a3b6899
I20260927 23:28:03.766202 2392 catalog_manager.cc:6565] ReplaceTablet: tablet 7767a32c07454fe6a92154429a3b6899 deleted and replaced by tablet 49883297d2b046b99f2ec1343c276ff9
I20260927 23:28:03.768385 2652 tablet_service.cc:1514] Processing DeleteTablet for tablet 7767a32c07454fe6a92154429a3b6899 with delete_type TABLET_DATA_DELETED (replaced by tablet 49883297d2b046b99f2ec1343c276ff9) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:03.769475 3341 ts_tablet_manager.cc:1918] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:03.769806 3341 log.cc:1196] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/95216fdc2e104a2e9fc4bc424bdbe5e9
I20260927 23:28:03.770449 3341 ts_tablet_manager.cc:1939] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:03.770895 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet 7767a32c07454fe6a92154429a3b6899 with delete_type TABLET_DATA_DELETED (replaced by tablet 49883297d2b046b99f2ec1343c276ff9) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:03.775655 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:03.778685 3336 ts_tablet_manager.cc:1905] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:03.776467 3370 leader_election.cc:288] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:03.781572 3397 raft_consensus.cc:482] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:03.782009 3397 raft_consensus.cc:504] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.787021 3070 tablet_bootstrap.cc:492] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:03.787449 3070 ts_tablet_manager.cc:1397] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.150s user 0.005s sys 0.005s
I20260927 23:28:03.787446 3397 leader_election.cc:288] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:03.788465 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9670df27c4404acda97661c69420b5cf" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:03.789759 2609 leader_election.cc:341] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:03.789893 3070 raft_consensus.cc:348] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.790576 3070 raft_consensus.cc:374] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:03.790829 3070 raft_consensus.cc:729] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:03.791464 3070 consensus_queue.cc:260] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.801575 2384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_e57880e3ffa64d3b876813b8c2246002" } modify_external_catalogs: true
I20260927 23:28:03.802410 2384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_e57880e3ffa64d3b876813b8c2246002" } modify_external_catalogs: true
I20260927 23:28:03.802557 2392 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_d8d7e47a77aa46818639a40ebd279a12" } new_table_name: "default.Table_2ca15ccb4f1f4b388143d9810dbb4ec6" modify_external_catalogs: true
I20260927 23:28:03.801831 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (table default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]) successfully deleted
W20260927 23:28:03.804431 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 39b3bf40c51646d38559f6852ebad6f0 because the tablet was not found. No further retry: Not found: Tablet not found: 39b3bf40c51646d38559f6852ebad6f0
W20260927 23:28:03.805182 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 2572268ca47346b0b78abc6a9ca7f908 because the tablet was not found. No further retry: Not found: Tablet not found: 2572268ca47346b0b78abc6a9ca7f908
W20260927 23:28:03.805796 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 7767a32c07454fe6a92154429a3b6899 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7767a32c07454fe6a92154429a3b6899 already in progress: creating tablet
I20260927 23:28:03.785233 2967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9670df27c4404acda97661c69420b5cf" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:03.813635 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7096e434ac9f4290ad58ba4f7e7c2513" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:03.813984 3070 ts_tablet_manager.cc:1428] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.026s user 0.000s sys 0.004s
I20260927 23:28:03.825891 2944 tablet_service.cc:1514] Processing DeleteTablet for tablet 7767a32c07454fe6a92154429a3b6899 with delete_type TABLET_DATA_DELETED (replaced by tablet 49883297d2b046b99f2ec1343c276ff9) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:03.826618 2761 leader_election.cc:302] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:03.827826 2824 raft_consensus.cc:3037] T fbfa55a66b8042ed8592fbff31e92d55 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:03.829581 3384 raft_consensus.cc:2726] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:03.834444 3407 tablet.cc:1722] T fbfa55a66b8042ed8592fbff31e92d55 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:03.835354 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 2572268ca47346b0b78abc6a9ca7f908 (table default.Table_2d889ed8e18e47a79f0541db93242386 [id=103d7c5e210c48fdafb8c199eb4cf676]) successfully deleted
W20260927 23:28:03.836045 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 517d7bfc225a4372bfd79682a550a0d1 because the tablet was not found. No further retry: Not found: Tablet not found: 517d7bfc225a4372bfd79682a550a0d1
W20260927 23:28:03.837116 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 39b3bf40c51646d38559f6852ebad6f0 because the tablet was not found. No further retry: Not found: Tablet not found: 39b3bf40c51646d38559f6852ebad6f0
I20260927 23:28:03.838706 2390 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_72f38013370a483784dfabf8e4646d24" } new_table_name: "default.Table_ce63ee103a6c4ae49b4f49360d1e1195" modify_external_catalogs: true
I20260927 23:28:03.843743 3070 tablet_bootstrap.cc:492] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:03.851120 3405 tablet.cc:1722] T fbfa55a66b8042ed8592fbff31e92d55 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:03.858194 3167 tablet_bootstrap.cc:492] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:03.858592 3167 ts_tablet_manager.cc:1397] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.175s user 0.026s sys 0.020s
I20260927 23:28:03.860741 3167 raft_consensus.cc:348] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.876844 3167 raft_consensus.cc:374] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:03.877108 3167 raft_consensus.cc:729] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:03.877913 3335 ts_tablet_manager.cc:1918] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:03.888165 3167 consensus_queue.cc:260] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.895453 3167 ts_tablet_manager.cc:1428] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.037s user 0.006s sys 0.000s
I20260927 23:28:03.896509 3167 tablet_bootstrap.cc:492] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
W20260927 23:28:03.878036 2609 consensus_peers.cc:487] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:03.907764 3335 log.cc:1196] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/5efbd0211cf3408cb5ea9640d8da6694
I20260927 23:28:03.878472 3070 tablet_bootstrap.cc:654] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:03.909036 3335 ts_tablet_manager.cc:1939] T 5efbd0211cf3408cb5ea9640d8da6694 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:03.809311 2967 raft_consensus.cc:2454] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:03.916604 3372 raft_consensus.cc:482] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:03.917106 3372 raft_consensus.cc:504] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:03.922250 3335 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 96) took 1006 ms. Trace:
I20260927 23:28:03.922591 3335 rpcz_store.cc:276] 0927 23:28:02.914867 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.915054 (+ 187us) service_pool.cc:227] Handling call
0927 23:28:03.448875 (+533821us) ts_tablet_manager.cc:1145] Deleting tablet 5efbd0211cf3408cb5ea9640d8da6694
0927 23:28:03.448929 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:03.659215 (+210286us) spinlock_profiling.cc:243] Waited 61.7 ms on lock 0x7b5400140d54. stack: 00007fe1d7372033 00007fe1d7371f30 00007fe1d6ef3534 00000000004db2b4 00000000004db249 00007fe1de27e471 00007fe1de301352 00007fe1de71726c 00007fe1df324cfa 00007fe1df354e10 00007fe1df33be0d 00007fe1df33bdb9 00007fe1df33bd49 00007fe1df33bd11 00007fe1df33af3d 0000000000580784
0927 23:28:03.799820 (+140605us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:03.877688 (+ 77868us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:03.920712 (+ 43024us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":7929,"spinlock_wait_cycles":141848320,"tablet-delete.queue_time_us":533081}
I20260927 23:28:03.923594 3372 leader_election.cc:288] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:03.924119 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 5efbd0211cf3408cb5ea9640d8da6694 (table default.Table_7c86d4308c504c4ba092bb6634326544 [id=0aa0174883974249b1120d0f02be108d]) successfully deleted
I20260927 23:28:03.925443 3167 tablet_bootstrap.cc:654] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:03.927281 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 2572268ca47346b0b78abc6a9ca7f908 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:59 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:03.934253 3335 tablet_replica.cc:330] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:03.936172 3415 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:44952 (request call id 79) took 1082 ms. Trace:
I20260927 23:28:03.936924 3335 raft_consensus.cc:2229] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:03.937335 3335 raft_consensus.cc:2258] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:03.938431 3070 tablet_bootstrap.cc:492] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:03.938951 3070 ts_tablet_manager.cc:1397] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.095s user 0.012s sys 0.004s
I20260927 23:28:03.943490 2969 raft_consensus.cc:2454] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:03.947994 3335 ts_tablet_manager.cc:1905] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:03.949165 2393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 16) took 1096 ms (client timeout 1000 ms). Trace:
I20260927 23:28:03.956248 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04f670db808c4e289c655679da9d2d9d" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:03.956985 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_0ec64b963d5643a291b169781879ee55"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:03.958369 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0ec64b963d5643a291b169781879ee55 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:03.960526 2608 leader_election.cc:302] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:03.961586 3397 raft_consensus.cc:2781] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:03.961887 3397 raft_consensus.cc:482] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:03.962199 3397 raft_consensus.cc:3037] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:03.962392 2907 leader_election.cc:341] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:03.963261 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04f670db808c4e289c655679da9d2d9d" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:03.964624 2906 leader_election.cc:341] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:03.965036 2906 leader_election.cc:302] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:03.966310 3370 raft_consensus.cc:2726] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:03.967907 2388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 3acdfe7faa1a4d5d8e340f09f1cae93e
I20260927 23:28:03.969148 2388 catalog_manager.cc:6565] ReplaceTablet: tablet 3acdfe7faa1a4d5d8e340f09f1cae93e deleted and replaced by tablet f58349edf0db40f98951e116fc2582b0
I20260927 23:28:03.971856 2608 leader_election.cc:302] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:03.972445 2608 raft_consensus.cc:2670] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260927 23:28:03.977190 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_4694990a288d48d5a3163f6c981c7bd5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:03.979689 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_9f465f0d2ddc47b49fafeebc89e141bf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:03.981051 2652 tablet_service.cc:1514] Processing DeleteTablet for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e with delete_type TABLET_DATA_DELETED (replaced by tablet f58349edf0db40f98951e116fc2582b0) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:03.987691 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e with delete_type TABLET_DATA_DELETED (replaced by tablet f58349edf0db40f98951e116fc2582b0) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:03.993016 3397 raft_consensus.cc:504] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:03.936549 3415 rpcz_store.cc:276] 0927 23:28:03.164875 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.204433 (+ 39558us) service_pool.cc:227] Handling call
0927 23:28:03.936102 (+731669us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:03.275888 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0927 23:28:03.288716 (+ 12828us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0927 23:28:03.288761 (+ 45us) alter_schema_op.cc:65] Acquired schema lock
0927 23:28:03.288839 (+ 78us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0927 23:28:03.288944 (+ 105us) alter_schema_op.cc:126] START. Timestamp: 1790551683288851
0927 23:28:03.288977 (+ 33us) op_driver.cc:331] REPLICATION: starting
0927 23:28:03.290281 (+ 1304us) log.cc:842] Serialized 173 byte log entry
0927 23:28:03.502874 (+212593us) spinlock_profiling.cc:243] Waited 44.4 ms on lock 0x7b200007c378. stack: 00007f5a8686c033 00007f5a8686bf30 00007f5a863ed534 00000000004db2b4 00000000004db249 00007f5a8d778471 00007f5a8d7d31f5 00007f5a8d7ef1b1 00007f5a8dc50ade 00007f5a8dc502b8 00007f5a8dc54ba1 00007f5a8dc54b59 00007f5a8dc54ae9 00007f5a8dc54ab1 00007f5a8dc53dad 0000000000580784
0927 23:28:03.602294 (+ 99420us) op_driver.cc:444] REPLICATION: finished
0927 23:28:03.651029 (+ 48735us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0927 23:28:03.652207 (+ 1178us) log.cc:842] Serialized 5 byte log entry
0927 23:28:03.830996 (+178789us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:03.917981 (+ 86985us) log.cc:842] Serialized 14 byte log entry
0927 23:28:03.926302 (+ 8321us) alter_schema_op.cc:71] Released schema lock
0927 23:28:03.926896 (+ 594us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":48581,"mutex_wait_us":1408,"prepare.queue_time_us":18726,"prepare.run_cpu_time_us":13200,"prepare.run_wall_time_us":235186,"raft.queue_time_us":377,"raft.run_cpu_time_us":51128,"raft.run_wall_time_us":309381,"replication_time_us":313236,"spinlock_wait_cycles":102011264,"thread_start_us":385527,"threads_started":2,"wal-append.queue_time_us":249980}]]}
I20260927 23:28:04.000815 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e with delete_type TABLET_DATA_DELETED (replaced by tablet f58349edf0db40f98951e116fc2582b0) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:04.002033 3341 tablet_replica.cc:330] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:04.002766 3341 raft_consensus.cc:2229] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:04.003288 3341 raft_consensus.cc:2258] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:04.005676 3341 ts_tablet_manager.cc:1905] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:03.952934 3070 raft_consensus.cc:348] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.006466 3070 raft_consensus.cc:374] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.006779 3070 raft_consensus.cc:729] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.007642 3070 consensus_queue.cc:260] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:28:04.014308 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4694990a288d48d5a3163f6c981c7bd5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:03.949522 2393 rpcz_store.cc:269] 0927 23:28:02.837387 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.837530 (+ 143us) service_pool.cc:227] Handling call
0927 23:28:02.888192 (+ 50662us) spinlock_profiling.cc:243] Waited 46 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:03.933287 (+1045095us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:03.700843 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:03.701210 (+ 367us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:03.701232 (+ 22us) write_op.cc:435] Acquired schema lock
0927 23:28:03.701246 (+ 14us) tablet.cc:660] Decoding operations
0927 23:28:03.701489 (+ 243us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:03.701524 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:03.701547 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:03.701672 (+ 125us) tablet.cc:699] Row locks acquired
0927 23:28:03.701686 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:03.701780 (+ 94us) write_op.cc:270] Start()
0927 23:28:03.701850 (+ 70us) write_op.cc:276] Timestamp: P: 1790551683683162 usec, L: 0
0927 23:28:03.701866 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:03.702299 (+ 433us) log.cc:842] Serialized 524 byte log entry
0927 23:28:03.764738 (+ 62439us) op_driver.cc:444] REPLICATION: finished
0927 23:28:03.766286 (+ 1548us) write_op.cc:301] APPLY: starting
0927 23:28:03.766337 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:03.766428 (+ 91us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:03.766526 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:03.766738 (+ 212us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:03.766887 (+ 149us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:03.766932 (+ 45us) write_op.cc:312] APPLY: finished
0927 23:28:03.767086 (+ 154us) log.cc:842] Serialized 23 byte log entry
0927 23:28:03.767217 (+ 131us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:03.767269 (+ 52us) write_op.cc:454] Released schema lock
0927 23:28:03.767296 (+ 27us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:03.767458 (+ 162us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:03.767484 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":25,"spinlock_wait_cycles":105732992,"child_traces":[["op",{"apply.queue_time_us":1393,"apply.run_cpu_time_us":1339,"apply.run_wall_time_us":1338,"num_ops":1,"prepare.queue_time_us":34567,"prepare.run_cpu_time_us":1889,"prepare.run_wall_time_us":56600,"raft.queue_time_us":1827,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":244,"replication_time_us":62811}]]}
W20260927 23:28:04.013872 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f465f0d2ddc47b49fafeebc89e141bf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:03.976044 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet fbfa55a66b8042ed8592fbff31e92d55 (table default.Table_2b44b865509b414ba9c0c667dd764097 [id=3b845f646d7f4af99b7889442b4d7d0f]). Expected version 1 got 0
I20260927 23:28:04.036149 3072 tablet_bootstrap.cc:492] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:04.044612 3072 ts_tablet_manager.cc:1397] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.551s user 0.017s sys 0.032s
I20260927 23:28:04.046888 3072 raft_consensus.cc:348] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.047493 3072 raft_consensus.cc:374] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.047755 3072 raft_consensus.cc:729] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.048410 3415 tablet.cc:1722] T 43096d258a654532906f59d581f5a5d6 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:04.049430 3405 tablet.cc:1722] T 43096d258a654532906f59d581f5a5d6 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:04.049712 3384 raft_consensus.cc:482] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:04.050806 3384 raft_consensus.cc:504] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.052788 3384 leader_election.cc:288] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:04.057725 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc1c0da6ea5c4a09a71555f37becae33" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:28:04.058034 2390 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2d0d8a82c5234880ab8396b1777e36c8 has no consensus state
I20260927 23:28:04.063442 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc1c0da6ea5c4a09a71555f37becae33" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:04.068620 2763 leader_election.cc:302] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:04.069732 3384 raft_consensus.cc:2726] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:04.072943 3167 tablet_bootstrap.cc:492] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:04.073309 3167 ts_tablet_manager.cc:1397] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.177s user 0.010s sys 0.009s
I20260927 23:28:04.075400 3167 raft_consensus.cc:348] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.076010 3167 raft_consensus.cc:374] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.076078 3397 leader_election.cc:288] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:04.076925 3336 ts_tablet_manager.cc:1918] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:04.078120 3336 log.cc:1196] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/828f870365244777b59e8a0756d7d4c9
I20260927 23:28:04.079114 3336 ts_tablet_manager.cc:1939] T 828f870365244777b59e8a0756d7d4c9 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:04.082957 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9670df27c4404acda97661c69420b5cf" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:04.096520 3072 consensus_queue.cc:260] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.097990 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_9e2be4d6dccd46449e29c796e22c2011"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:04.103333 3072 ts_tablet_manager.cc:1428] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.058s user 0.005s sys 0.000s
I20260927 23:28:04.084992 3070 ts_tablet_manager.cc:1428] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.143s user 0.003s sys 0.002s
I20260927 23:28:04.104689 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 74) took 1260 ms. Trace:
I20260927 23:28:04.105006 3336 rpcz_store.cc:276] 0927 23:28:02.844288 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.844402 (+ 114us) service_pool.cc:227] Handling call
0927 23:28:03.716503 (+872101us) ts_tablet_manager.cc:1145] Deleting tablet 828f870365244777b59e8a0756d7d4c9
0927 23:28:03.716549 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:03.936825 (+220276us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:04.076680 (+139855us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:04.104633 (+ 27953us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22417,"tablet-delete.queue_time_us":805772}
I20260927 23:28:04.076251 3167 raft_consensus.cc:729] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.110641 3070 tablet_bootstrap.cc:492] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
W20260927 23:28:04.116894 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 99213.6us
I20260927 23:28:04.119807 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9670df27c4404acda97661c69420b5cf" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:04.120429 2969 raft_consensus.cc:3037] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:04.106215 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 828f870365244777b59e8a0756d7d4c9 (table default.Table_25f0e3650873407a8b3cbd8c8e0c86af [id=48f252a9cdb34eb1b452540f4186285c]) successfully deleted
I20260927 23:28:04.127184 2969 raft_consensus.cc:2454] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:04.133263 3250 raft_consensus.cc:482] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:04.133844 3250 raft_consensus.cc:504] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.137287 3420 tablet.cc:1722] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:04.137682 2608 leader_election.cc:302] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:04.138437 3112 raft_consensus.cc:2781] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:04.138809 3112 raft_consensus.cc:686] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:04.139432 3112 consensus_queue.cc:237] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
W20260927 23:28:04.147699 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9e2be4d6dccd46449e29c796e22c2011 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:04.136938 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80bff324902498fa5c04fe8cbc05f9d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:04.154071 2609 leader_election.cc:341] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:04.156097 3335 ts_tablet_manager.cc:1918] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:04.161993 3335 log.cc:1196] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/9cce47b050b445dfba7be5ea3b6a8636
I20260927 23:28:04.162726 3335 ts_tablet_manager.cc:1939] T 9cce47b050b445dfba7be5ea3b6a8636 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:04.154855 3167 consensus_queue.cc:260] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.166270 3167 ts_tablet_manager.cc:1428] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.093s user 0.006s sys 0.000s
I20260927 23:28:04.168001 3167 tablet_bootstrap.cc:492] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:04.175518 3072 tablet_bootstrap.cc:492] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:04.179139 3341 ts_tablet_manager.cc:1918] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:04.179713 3341 log.cc:1196] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/3acdfe7faa1a4d5d8e340f09f1cae93e
I20260927 23:28:04.178143 2824 raft_consensus.cc:3037] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:04.180739 3336 tablet_replica.cc:330] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
W20260927 23:28:04.188398 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 50807.5us
I20260927 23:28:04.198642 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80bff324902498fa5c04fe8cbc05f9d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:04.199280 2969 raft_consensus.cc:2454] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:04.200246 3336 raft_consensus.cc:2229] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:04.201097 3336 pending_rounds.cc:62] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d: Trying to abort 1 pending ops.
I20260927 23:28:04.201373 3336 pending_rounds.cc:68] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099696714838016 op_type: NO_OP noop_request { }
I20260927 23:28:04.201824 3336 raft_consensus.cc:2866] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:28:04.202623 3336 raft_consensus.cc:2258] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:04.204787 3336 ts_tablet_manager.cc:1905] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:04.205463 3335 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 100) took 1099 ms. Trace:
I20260927 23:28:04.205933 2824 raft_consensus.cc:2454] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:04.205830 3335 rpcz_store.cc:276] 0927 23:28:03.105750 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.105916 (+ 166us) service_pool.cc:227] Handling call
0927 23:28:03.933761 (+827845us) ts_tablet_manager.cc:1145] Deleting tablet 9cce47b050b445dfba7be5ea3b6a8636
0927 23:28:03.933801 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:04.019509 (+ 85708us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:04.090834 (+ 71325us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:04.205401 (+114567us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":827220}
I20260927 23:28:04.207347 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 9cce47b050b445dfba7be5ea3b6a8636 (table default.Table_92a19ba603ef488c972786364cf49c70 [id=b12aae65f933476ab57974d4b328a266]) successfully deleted
I20260927 23:28:04.208261 3335 tablet_replica.cc:330] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:04.208986 3335 raft_consensus.cc:2229] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:04.209473 3335 raft_consensus.cc:2258] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:04.209582 2387 master_service.cc:946] ReplaceTablet: received request to replace tablet 7767a32c07454fe6a92154429a3b6899 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:04.211081 3335 ts_tablet_manager.cc:1905] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:04.221906 2378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2d0d8a82c5234880ab8396b1777e36c8 has no consensus state
I20260927 23:28:04.219062 3341 ts_tablet_manager.cc:1939] T 3acdfe7faa1a4d5d8e340f09f1cae93e P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:04.227912 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 3acdfe7faa1a4d5d8e340f09f1cae93e (table default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2]) successfully deleted
I20260927 23:28:04.223699 3072 tablet_bootstrap.cc:654] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.211311 3250 leader_election.cc:288] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:04.214449 3070 tablet_bootstrap.cc:654] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.215929 3407 tablet.cc:1722] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:04.237540 2385 master_service.cc:946] ReplaceTablet: received request to replace tablet 410294c28c49404e9c547b27111712eb from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:04.242941 3167 tablet_bootstrap.cc:654] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.219521 2608 leader_election.cc:302] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:04.250105 3438 raft_consensus.cc:2781] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:04.250468 3438 raft_consensus.cc:482] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:04.250735 3438 raft_consensus.cc:3037] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:04.258265 3438 raft_consensus.cc:504] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.260818 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80bff324902498fa5c04fe8cbc05f9d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:04.261373 2969 raft_consensus.cc:3037] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:04.261380 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80bff324902498fa5c04fe8cbc05f9d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:04.262784 2609 leader_election.cc:341] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:04.266184 3070 tablet_bootstrap.cc:492] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:04.266571 3070 ts_tablet_manager.cc:1397] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.156s user 0.013s sys 0.001s
I20260927 23:28:04.268733 3070 raft_consensus.cc:348] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:04.269376 3070 raft_consensus.cc:374] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.269615 3070 raft_consensus.cc:729] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.270190 3070 consensus_queue.cc:260] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:04.278667 2969 raft_consensus.cc:2454] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:04.280396 3438 leader_election.cc:288] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:04.274194 3407 tablet.cc:1722] T 43096d258a654532906f59d581f5a5d6 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:04.282007 2608 leader_election.cc:302] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:04.283787 3438 raft_consensus.cc:2781] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:04.284509 3438 raft_consensus.cc:686] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:04.284792 3070 ts_tablet_manager.cc:1428] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.018s user 0.006s sys 0.001s
I20260927 23:28:04.285346 3438 consensus_queue.cc:237] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.293915 3167 tablet_bootstrap.cc:492] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:04.294461 3167 ts_tablet_manager.cc:1397] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.127s user 0.011s sys 0.005s
I20260927 23:28:04.291481 3070 tablet_bootstrap.cc:492] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:04.296530 3167 raft_consensus.cc:348] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.297142 3167 raft_consensus.cc:374] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.297377 3167 raft_consensus.cc:729] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.298668 3167 consensus_queue.cc:260] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.303320 3070 tablet_bootstrap.cc:654] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.308539 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 7f530db3b52944f6a283ee9e2bf66828 (DEFAULT_TABLE table=default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.309636 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f530db3b52944f6a283ee9e2bf66828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.311197 3167 ts_tablet_manager.cc:1428] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.016s user 0.004s sys 0.000s
I20260927 23:28:04.312922 3072 tablet_bootstrap.cc:492] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:04.313395 3072 ts_tablet_manager.cc:1397] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.138s user 0.009s sys 0.000s
I20260927 23:28:04.319947 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 7f530db3b52944f6a283ee9e2bf66828 (DEFAULT_TABLE table=default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.321229 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f530db3b52944f6a283ee9e2bf66828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.321511 3167 tablet_bootstrap.cc:492] T 1d299ca52cfc493499b5eaf0acdeb5f9 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:04.322578 2803 tablet_service.cc:1467] Processing CreateTablet for tablet 3c6e596517324300be30df8506a18904 (DEFAULT_TABLE table=default.Table_585d1f792a1b42e7ae86d5ecb5edaff2 [id=ff09406f8316449ead4cffa47a2095b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.323670 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c6e596517324300be30df8506a18904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.331583 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 7f530db3b52944f6a283ee9e2bf66828 (DEFAULT_TABLE table=default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.332175 3167 tablet_bootstrap.cc:654] T 1d299ca52cfc493499b5eaf0acdeb5f9 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.332582 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f530db3b52944f6a283ee9e2bf66828. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.346625 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 3c6e596517324300be30df8506a18904 (DEFAULT_TABLE table=default.Table_585d1f792a1b42e7ae86d5ecb5edaff2 [id=ff09406f8316449ead4cffa47a2095b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.347211 2944 tablet_service.cc:1467] Processing CreateTablet for tablet 3c6e596517324300be30df8506a18904 (DEFAULT_TABLE table=default.Table_585d1f792a1b42e7ae86d5ecb5edaff2 [id=ff09406f8316449ead4cffa47a2095b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.347589 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c6e596517324300be30df8506a18904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.348387 2944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c6e596517324300be30df8506a18904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.348819 3072 raft_consensus.cc:348] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.349486 3072 raft_consensus.cc:374] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.349713 3072 raft_consensus.cc:729] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.350286 3072 consensus_queue.cc:260] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.352259 3072 ts_tablet_manager.cc:1428] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.039s user 0.005s sys 0.000s
I20260927 23:28:04.353106 3072 tablet_bootstrap.cc:492] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:04.369592 3368 raft_consensus.cc:482] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:04.370004 3368 raft_consensus.cc:504] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.371673 3368 leader_election.cc:288] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:04.378204 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1b6b75dad24c609a10693728ba8e20" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:04.379513 2907 leader_election.cc:341] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:04.381537 2393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_5f156a98d2694e9aa22d5bcce0e0890d" } new_table_name: "default.Table_08f6c5348e2b41feba84b48a70ac7a8c" modify_external_catalogs: true
I20260927 23:28:04.387997 3072 tablet_bootstrap.cc:654] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.400132 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae (DEFAULT_TABLE table=default.Table_f1571426f866470d94cfe827bd30ba79 [id=0bcc62622acc4fb7a60d137f1b75265d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.400131 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae (DEFAULT_TABLE table=default.Table_f1571426f866470d94cfe827bd30ba79 [id=0bcc62622acc4fb7a60d137f1b75265d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.401141 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.401171 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.409708 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae (DEFAULT_TABLE table=default.Table_f1571426f866470d94cfe827bd30ba79 [id=0bcc62622acc4fb7a60d137f1b75265d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.410796 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.412428 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2a39980f0c46c68becefacad890ab7 (DEFAULT_TABLE table=default.Table_69a2163339974120a45253fcfdb43e98 [id=705a393152f446888d042c40aab17f3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.413414 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2a39980f0c46c68becefacad890ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.417719 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_57fe58bb7f964bdf9bbfcd03dd684f66"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:04.419076 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_57fe58bb7f964bdf9bbfcd03dd684f66 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:04.421149 3384 raft_consensus.cc:482] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:04.421552 3384 raft_consensus.cc:504] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.423280 3384 leader_election.cc:288] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:04.426429 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_1322a4efff174b9e8c09d2be9f8e4570"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:04.427865 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1322a4efff174b9e8c09d2be9f8e4570 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:04.429292 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66f2842218fc42e5b1b83314e5f1a158" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:04.433478 3336 ts_tablet_manager.cc:1918] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:04.433904 3336 log.cc:1196] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/7096e434ac9f4290ad58ba4f7e7c2513
I20260927 23:28:04.434630 3336 ts_tablet_manager.cc:1939] T 7096e434ac9f4290ad58ba4f7e7c2513 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:04.435324 3438 raft_consensus.cc:482] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:04.435819 3438 raft_consensus.cc:504] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.438580 3438 leader_election.cc:288] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:04.440537 3335 ts_tablet_manager.cc:1918] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:04.440882 3335 log.cc:1196] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/95216fdc2e104a2e9fc4bc424bdbe5e9
I20260927 23:28:04.441469 3335 ts_tablet_manager.cc:1939] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:04.444198 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac49c5b79e347158621dda52f79e912" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:04.444805 2972 raft_consensus.cc:2454] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:04.445969 2608 leader_election.cc:302] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:04.446672 3438 raft_consensus.cc:2781] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:04.446827 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 75) took 1601 ms. Trace:
I20260927 23:28:04.446990 3438 raft_consensus.cc:482] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:04.447290 3438 raft_consensus.cc:3037] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:04.454213 3167 tablet_bootstrap.cc:492] T 1d299ca52cfc493499b5eaf0acdeb5f9 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:04.454692 3167 ts_tablet_manager.cc:1397] T 1d299ca52cfc493499b5eaf0acdeb5f9 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.133s user 0.012s sys 0.005s
I20260927 23:28:04.456630 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1b6b75dad24c609a10693728ba8e20" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:04.457655 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac49c5b79e347158621dda52f79e912" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:04.458678 2609 leader_election.cc:341] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:04.459975 2906 consensus_peers.cc:487] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3acdfe7faa1a4d5d8e340f09f1cae93e. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:04.460810 2906 leader_election.cc:341] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:04.461167 2906 leader_election.cc:302] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:04.461334 2763 leader_election.cc:341] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 66f2842218fc42e5b1b83314e5f1a158
I20260927 23:28:04.461859 2763 leader_election.cc:302] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:04.462529 3311 raft_consensus.cc:2726] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:04.462800 3384 raft_consensus.cc:2726] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:04.463510 3438 raft_consensus.cc:504] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.466702 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac49c5b79e347158621dda52f79e912" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:04.467356 2972 raft_consensus.cc:3037] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:04.477169 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac49c5b79e347158621dda52f79e912" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:04.477402 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (table default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]) successfully deleted
I20260927 23:28:04.447139 3336 rpcz_store.cc:276] 0927 23:28:02.845009 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.845108 (+ 99us) service_pool.cc:227] Handling call
0927 23:28:04.180555 (+1335447us) ts_tablet_manager.cc:1145] Deleting tablet 7096e434ac9f4290ad58ba4f7e7c2513
0927 23:28:04.180620 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:04.359101 (+178481us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:04.433257 (+ 74156us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:04.446774 (+ 13517us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6827,"tablet-delete.queue_time_us":1269690}
W20260927 23:28:04.478452 2609 leader_election.cc:341] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:04.478760 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 7096e434ac9f4290ad58ba4f7e7c2513 (table default.Table_3963531cde4641e785245322c0a23fab [id=1cac6a4a98734d4abf99d93891e1f077]) successfully deleted
I20260927 23:28:04.479530 3438 leader_election.cc:288] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:28:04.480118 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 45dd55b5a9d94ff0b84a799174ab5359 because the tablet was not found. No further retry: Not found: Tablet not found: 45dd55b5a9d94ff0b84a799174ab5359
I20260927 23:28:04.480134 2972 raft_consensus.cc:2454] T fac49c5b79e347158621dda52f79e912 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:04.481098 3336 tablet_replica.cc:330] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:04.481092 2608 leader_election.cc:302] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:04.481997 3438 raft_consensus.cc:2781] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:04.482724 2385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 410294c28c49404e9c547b27111712eb
W20260927 23:28:04.483379 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 66f2842218fc42e5b1b83314e5f1a158 because the tablet was not found. No further retry: Not found: Tablet not found: 66f2842218fc42e5b1b83314e5f1a158
I20260927 23:28:04.483883 3438 raft_consensus.cc:686] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:04.484004 2385 catalog_manager.cc:6565] ReplaceTablet: tablet 410294c28c49404e9c547b27111712eb deleted and replaced by tablet 8cd0972f635d4979abf5fd30e9b02b1d
I20260927 23:28:04.484918 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_27f9b741e6184aa0bb0f95975292a4ae"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:04.484970 2652 tablet_service.cc:1514] Processing DeleteTablet for tablet 410294c28c49404e9c547b27111712eb with delete_type TABLET_DATA_DELETED (replaced by tablet 8cd0972f635d4979abf5fd30e9b02b1d) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:04.485428 2805 tablet_service.cc:1514] Processing DeleteTablet for tablet 410294c28c49404e9c547b27111712eb with delete_type TABLET_DATA_DELETED (replaced by tablet 8cd0972f635d4979abf5fd30e9b02b1d) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:04.484650 3438 consensus_queue.cc:237] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.486011 3341 tablet_replica.cc:330] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
W20260927 23:28:04.486347 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_27f9b741e6184aa0bb0f95975292a4ae in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:04.486641 3341 raft_consensus.cc:2229] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:04.487112 3341 raft_consensus.cc:2258] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:04.488982 3341 ts_tablet_manager.cc:1905] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:04.490744 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:04.491466 2391 catalog_manager.cc:5495] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:04.495153 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1d299ca52cfc493499b5eaf0acdeb5f9 (table default.Table_bf18dd7dd7984680bf41384772c15e61 [id=1af28dc40a8d408ba31c695e8563efc2]). Expected version 1 got 0
I20260927 23:28:04.496137 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet fbfa55a66b8042ed8592fbff31e92d55 (table default.Table_2b44b865509b414ba9c0c667dd764097 [id=3b845f646d7f4af99b7889442b4d7d0f]). Expected version 1 got 0
I20260927 23:28:04.496759 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet cc1c0da6ea5c4a09a71555f37becae33 (table default.Table_29404653bfbb4e5d8192bcb604084df0 [id=baad95258e1a4358a58abc3fc1cc3737]). Expected version 1 got 0
I20260927 23:28:04.504918 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2a39980f0c46c68becefacad890ab7 (DEFAULT_TABLE table=default.Table_69a2163339974120a45253fcfdb43e98 [id=705a393152f446888d042c40aab17f3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.505853 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2a39980f0c46c68becefacad890ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.505134 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 0e2a39980f0c46c68becefacad890ab7 (DEFAULT_TABLE table=default.Table_69a2163339974120a45253fcfdb43e98 [id=705a393152f446888d042c40aab17f3c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.506871 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e2a39980f0c46c68becefacad890ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.505869 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet 410294c28c49404e9c547b27111712eb with delete_type TABLET_DATA_DELETED (replaced by tablet 8cd0972f635d4979abf5fd30e9b02b1d) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:04.514582 3070 tablet_bootstrap.cc:492] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:04.514968 3070 ts_tablet_manager.cc:1397] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.230s user 0.035s sys 0.044s
I20260927 23:28:04.516399 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 1912a82ff46f4f9aaa24b7621cc3995d (DEFAULT_TABLE table=default.Table_678fca7e3fe641c39d8cdc436fc85d38 [id=931a9b7ab80b47268596521d6be6a458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.517452 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1912a82ff46f4f9aaa24b7621cc3995d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.519559 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 1912a82ff46f4f9aaa24b7621cc3995d (DEFAULT_TABLE table=default.Table_678fca7e3fe641c39d8cdc436fc85d38 [id=931a9b7ab80b47268596521d6be6a458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.517093 3070 raft_consensus.cc:348] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:04.520486 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1912a82ff46f4f9aaa24b7621cc3995d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.456887 3167 raft_consensus.cc:348] T 1d299ca52cfc493499b5eaf0acdeb5f9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.520421 3070 raft_consensus.cc:374] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.520715 3167 raft_consensus.cc:374] T 1d299ca52cfc493499b5eaf0acdeb5f9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.520743 3070 raft_consensus.cc:729] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.521366 3070 consensus_queue.cc:260] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:04.523504 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 1912a82ff46f4f9aaa24b7621cc3995d (DEFAULT_TABLE table=default.Table_678fca7e3fe641c39d8cdc436fc85d38 [id=931a9b7ab80b47268596521d6be6a458]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.525094 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1912a82ff46f4f9aaa24b7621cc3995d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.531270 3336 raft_consensus.cc:2229] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:04.531808 3336 raft_consensus.cc:2258] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:04.520934 3167 raft_consensus.cc:729] T 1d299ca52cfc493499b5eaf0acdeb5f9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.535175 3335 tablet_replica.cc:330] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
W20260927 23:28:04.535157 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 828f870365244777b59e8a0756d7d4c9 because the tablet was not found. No further retry: Not found: Tablet not found: 828f870365244777b59e8a0756d7d4c9
I20260927 23:28:04.535897 3335 raft_consensus.cc:2229] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260927 23:28:04.536226 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 7767a32c07454fe6a92154429a3b6899 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7767a32c07454fe6a92154429a3b6899 already in progress: creating tablet
I20260927 23:28:04.532763 3167 consensus_queue.cc:260] T 1d299ca52cfc493499b5eaf0acdeb5f9 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.537153 3070 ts_tablet_manager.cc:1428] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260927 23:28:04.538405 3070 tablet_bootstrap.cc:492] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:04.542127 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 92b7eb04f8ff4726b4e9fe3d048ddafc (DEFAULT_TABLE table=default.Table_2d9f46546b8a45fd8747186ee6775db1 [id=3c03820320244cd9ad9a8bf7b4631e47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.542393 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 92b7eb04f8ff4726b4e9fe3d048ddafc (DEFAULT_TABLE table=default.Table_2d9f46546b8a45fd8747186ee6775db1 [id=3c03820320244cd9ad9a8bf7b4631e47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.543212 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b7eb04f8ff4726b4e9fe3d048ddafc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.543339 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b7eb04f8ff4726b4e9fe3d048ddafc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.543617 3070 tablet_bootstrap.cc:654] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.551187 3341 ts_tablet_manager.cc:1918] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260927 23:28:04.551517 3341 log.cc:1196] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/410294c28c49404e9c547b27111712eb
I20260927 23:28:04.552528 3167 ts_tablet_manager.cc:1428] T 1d299ca52cfc493499b5eaf0acdeb5f9 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.098s user 0.005s sys 0.000s
W20260927 23:28:04.565966 2378 consensus_peers.cc:487] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f -> Peer 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571): Couldn't send request to peer 0cdd64d48a684d5ebccc2e2e8a4e3150. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260927 23:28:04.567201 2806 tablet_service.cc:1467] Processing CreateTablet for tablet fd0115095a82489a8aeb941efcc8a979 (DEFAULT_TABLE table=default.Table_63f9354acd7d4ca3ad92ab1edf9d7174 [id=0a5810627a964e7caf66c69e7c3ec9b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.568220 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0115095a82489a8aeb941efcc8a979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.570667 3167 tablet_bootstrap.cc:492] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:04.574698 3335 raft_consensus.cc:2258] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:04.547379 3072 tablet_bootstrap.cc:492] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:04.557634 2944 tablet_service.cc:1467] Processing CreateTablet for tablet 92b7eb04f8ff4726b4e9fe3d048ddafc (DEFAULT_TABLE table=default.Table_2d9f46546b8a45fd8747186ee6775db1 [id=3c03820320244cd9ad9a8bf7b4631e47]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.576864 3072 ts_tablet_manager.cc:1397] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.224s user 0.010s sys 0.000s
I20260927 23:28:04.577530 2944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b7eb04f8ff4726b4e9fe3d048ddafc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.578972 3072 raft_consensus.cc:348] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.579592 3072 raft_consensus.cc:374] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.579825 3072 raft_consensus.cc:729] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.580464 3072 consensus_queue.cc:260] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.587244 3072 ts_tablet_manager.cc:1428] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20260927 23:28:04.533803 3336 ts_tablet_manager.cc:1905] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:04.588420 3072 tablet_bootstrap.cc:492] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:04.594691 3335 ts_tablet_manager.cc:1905] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:04.597159 2648 tablet_service.cc:1467] Processing CreateTablet for tablet fd0115095a82489a8aeb941efcc8a979 (DEFAULT_TABLE table=default.Table_63f9354acd7d4ca3ad92ab1edf9d7174 [id=0a5810627a964e7caf66c69e7c3ec9b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.598291 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0115095a82489a8aeb941efcc8a979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.603488 3341 ts_tablet_manager.cc:1939] T 410294c28c49404e9c547b27111712eb P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:04.609568 3070 tablet_bootstrap.cc:492] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:04.609944 3070 ts_tablet_manager.cc:1397] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.072s user 0.007s sys 0.000s
I20260927 23:28:04.611972 3070 raft_consensus.cc:348] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:04.612722 3070 raft_consensus.cc:374] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.612962 3070 raft_consensus.cc:729] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.613601 3070 consensus_queue.cc:260] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:04.615566 3070 ts_tablet_manager.cc:1428] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 23:28:04.616355 3070 tablet_bootstrap.cc:492] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
W20260927 23:28:04.616629 2907 consensus_peers.cc:487] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:04.618680 2950 tablet_service.cc:1467] Processing CreateTablet for tablet fd0115095a82489a8aeb941efcc8a979 (DEFAULT_TABLE table=default.Table_63f9354acd7d4ca3ad92ab1edf9d7174 [id=0a5810627a964e7caf66c69e7c3ec9b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.619568 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd0115095a82489a8aeb941efcc8a979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.620155 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 (DEFAULT_TABLE table=default.Table_25f0e3650873407a8b3cbd8c8e0c86af [id=48f252a9cdb34eb1b452540f4186285c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.621146 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bf856a05dff4fb4943fe8e1d9b7f385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.625094 3070 tablet_bootstrap.cc:654] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.648213 3070 tablet_bootstrap.cc:492] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:04.648680 3070 ts_tablet_manager.cc:1397] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.000s
I20260927 23:28:04.649003 2384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 3200d6bd5cb24ff5b1bbd59af9353dbe
I20260927 23:28:04.650699 3070 raft_consensus.cc:348] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:04.651227 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:04.651307 3070 raft_consensus.cc:374] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.651532 3070 raft_consensus.cc:729] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.653219 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:04.654733 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_10714592f65b41f5ad17642613dc4537"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:04.656131 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_10714592f65b41f5ad17642613dc4537 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:04.652064 3070 consensus_queue.cc:260] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:04.666553 2388 master_service.cc:946] ReplaceTablet: received request to replace tablet a4277045ec344ad7a48db9b14bea5e7b from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:04.670390 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 410294c28c49404e9c547b27111712eb (table default.Table_813e5d484f8f462b86e9136a57eebacd [id=b215370f7b594ab28c3e2a34de2a6f16]) successfully deleted
I20260927 23:28:04.671207 3070 ts_tablet_manager.cc:1428] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260927 23:28:04.683329 3070 tablet_bootstrap.cc:492] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
W20260927 23:28:04.688139 2609 consensus_peers.cc:487] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 410294c28c49404e9c547b27111712eb. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:04.689620 3070 tablet_bootstrap.cc:654] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.690405 2803 tablet_service.cc:1514] Processing DeleteTablet for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:04.690961 3072 tablet_bootstrap.cc:654] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.695009 3335 ts_tablet_manager.cc:1918] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:04.695354 3335 log.cc:1196] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/3acdfe7faa1a4d5d8e340f09f1cae93e
I20260927 23:28:04.695997 3335 ts_tablet_manager.cc:1939] T 3acdfe7faa1a4d5d8e340f09f1cae93e P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
W20260927 23:28:04.697470 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3200d6bd5cb24ff5b1bbd59af9353dbe already in progress: creating tablet
I20260927 23:28:04.698192 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 3acdfe7faa1a4d5d8e340f09f1cae93e (table default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2]) successfully deleted
I20260927 23:28:04.700912 3335 tablet_replica.cc:330] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:04.703685 3335 raft_consensus.cc:2229] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:04.704583 3335 raft_consensus.cc:2258] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:04.725697 3335 ts_tablet_manager.cc:1905] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:04.727752 3336 ts_tablet_manager.cc:1918] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:04.730129 3070 tablet_bootstrap.cc:492] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:04.730538 3070 ts_tablet_manager.cc:1397] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.059s user 0.012s sys 0.005s
I20260927 23:28:04.733088 3070 raft_consensus.cc:348] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.733711 3070 raft_consensus.cc:374] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.733965 3070 raft_consensus.cc:729] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.734606 3070 consensus_queue.cc:260] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.738543 3070 ts_tablet_manager.cc:1428] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20260927 23:28:04.739319 3070 tablet_bootstrap.cc:492] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:04.743803 3335 ts_tablet_manager.cc:1918] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260927 23:28:04.744129 3335 log.cc:1196] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/410294c28c49404e9c547b27111712eb
I20260927 23:28:04.745468 3335 ts_tablet_manager.cc:1939] T 410294c28c49404e9c547b27111712eb P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:04.745767 3454 raft_consensus.cc:482] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:04.746254 3454 raft_consensus.cc:504] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.747617 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 410294c28c49404e9c547b27111712eb (table default.Table_813e5d484f8f462b86e9136a57eebacd [id=b215370f7b594ab28c3e2a34de2a6f16]) successfully deleted
I20260927 23:28:04.749708 3070 tablet_bootstrap.cc:654] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:04.750814 2763 leader_election.cc:341] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 9cce47b050b445dfba7be5ea3b6a8636
I20260927 23:28:04.748221 3454 leader_election.cc:288] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:04.755451 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3200d6bd5cb24ff5b1bbd59af9353dbe already in progress: creating tablet
I20260927 23:28:04.759236 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:04.749326 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cce47b050b445dfba7be5ea3b6a8636" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:04.762101 2972 raft_consensus.cc:2379] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:04.763300 2761 leader_election.cc:302] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:04.764086 3454 raft_consensus.cc:3037] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:04.760993 3336 log.cc:1196] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/a80bff324902498fa5c04fe8cbc05f9d
I20260927 23:28:04.766263 3072 tablet_bootstrap.cc:492] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:04.766630 3072 ts_tablet_manager.cc:1397] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.178s user 0.013s sys 0.000s
I20260927 23:28:04.766769 3336 ts_tablet_manager.cc:1939] T a80bff324902498fa5c04fe8cbc05f9d P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:04.768729 3072 raft_consensus.cc:348] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.769358 3072 raft_consensus.cc:374] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.769593 3072 raft_consensus.cc:729] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.771785 3454 raft_consensus.cc:2726] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:04.773037 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 77) took 1927 ms. Trace:
I20260927 23:28:04.773609 3336 rpcz_store.cc:276] 0927 23:28:02.846014 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.852459 (+ 6445us) service_pool.cc:227] Handling call
0927 23:28:04.480930 (+1628471us) ts_tablet_manager.cc:1145] Deleting tablet a80bff324902498fa5c04fe8cbc05f9d
0927 23:28:04.480979 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:04.688642 (+207663us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:04.727533 (+ 38891us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:04.772972 (+ 45439us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1569873}
I20260927 23:28:04.775038 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet a80bff324902498fa5c04fe8cbc05f9d (table default.Table_a5f98162932247a3ae2928ce03a691a4 [id=51372c7a0c1b4c9db9df0e9b0cbbe071]) successfully deleted
I20260927 23:28:04.775273 3336 tablet_replica.cc:330] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:04.776633 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 (DEFAULT_TABLE table=default.Table_25f0e3650873407a8b3cbd8c8e0c86af [id=48f252a9cdb34eb1b452540f4186285c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.777539 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bf856a05dff4fb4943fe8e1d9b7f385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.777485 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0d8a82c5234880ab8396b1777e36c8 (DEFAULT_TABLE table=default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.777788 3336 raft_consensus.cc:2229] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:04.778206 3336 raft_consensus.cc:2258] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:04.778523 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0d8a82c5234880ab8396b1777e36c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.780069 3336 ts_tablet_manager.cc:1905] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:04.775970 3072 consensus_queue.cc:260] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.789268 2803 tablet_service.cc:1467] Processing CreateTablet for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 (DEFAULT_TABLE table=default.Table_25f0e3650873407a8b3cbd8c8e0c86af [id=48f252a9cdb34eb1b452540f4186285c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.790445 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bf856a05dff4fb4943fe8e1d9b7f385. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.790876 3072 ts_tablet_manager.cc:1428] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.024s user 0.005s sys 0.001s
I20260927 23:28:04.794538 3072 tablet_bootstrap.cc:492] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:04.805872 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet cc1c0da6ea5c4a09a71555f37becae33 (table default.Table_29404653bfbb4e5d8192bcb604084df0 [id=baad95258e1a4358a58abc3fc1cc3737]). Expected version 1 got 0
I20260927 23:28:04.806989 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 1d299ca52cfc493499b5eaf0acdeb5f9 (table default.Table_bf18dd7dd7984680bf41384772c15e61 [id=1af28dc40a8d408ba31c695e8563efc2]). Expected version 1 got 0
I20260927 23:28:04.817975 3167 tablet_bootstrap.cc:654] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.837029 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0d8a82c5234880ab8396b1777e36c8 (DEFAULT_TABLE table=default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.838675 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0d8a82c5234880ab8396b1777e36c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.839298 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 1da38499af164d7fb5c1a8d65167ad24 (DEFAULT_TABLE table=default.Table_92a19ba603ef488c972786364cf49c70 [id=b12aae65f933476ab57974d4b328a266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.840143 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1da38499af164d7fb5c1a8d65167ad24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.846163 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 2d0d8a82c5234880ab8396b1777e36c8 (DEFAULT_TABLE table=default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.847076 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d0d8a82c5234880ab8396b1777e36c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.853992 2824 raft_consensus.cc:3037] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:04.858070 3072 tablet_bootstrap.cc:654] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.856923 2387 master_service.cc:946] ReplaceTablet: received request to replace tablet ac19099dd7bf4b669d504a654ee69482 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:04.864673 3336 ts_tablet_manager.cc:1918] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:04.867694 3336 log.cc:1196] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/45dd55b5a9d94ff0b84a799174ab5359
I20260927 23:28:04.868358 3336 ts_tablet_manager.cc:1939] T 45dd55b5a9d94ff0b84a799174ab5359 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:04.870357 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 78) took 2017 ms. Trace:
I20260927 23:28:04.870647 3336 rpcz_store.cc:276] 0927 23:28:02.853175 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:02.853432 (+ 257us) service_pool.cc:227] Handling call
0927 23:28:04.775114 (+1921682us) ts_tablet_manager.cc:1145] Deleting tablet 45dd55b5a9d94ff0b84a799174ab5359
0927 23:28:04.775166 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:04.802765 (+ 27599us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:04.864468 (+ 61703us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:04.870306 (+ 5838us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1863902}
I20260927 23:28:04.871691 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 45dd55b5a9d94ff0b84a799174ab5359 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]) successfully deleted
W20260927 23:28:04.874403 2907 consensus_peers.cc:487] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 95216fdc2e104a2e9fc4bc424bdbe5e9. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:04.875152 3336 tablet_replica.cc:330] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:04.875867 3336 raft_consensus.cc:2229] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:04.876545 2388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet a4277045ec344ad7a48db9b14bea5e7b
I20260927 23:28:04.877755 2388 catalog_manager.cc:6565] ReplaceTablet: tablet a4277045ec344ad7a48db9b14bea5e7b deleted and replaced by tablet 0a2f98dd905d40959dc676f94a4a595b
I20260927 23:28:04.878695 2388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_5924c026ff234c36b073a7fce24b05e6" } modify_external_catalogs: true
I20260927 23:28:04.879184 2388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_5924c026ff234c36b073a7fce24b05e6" } modify_external_catalogs: true
I20260927 23:28:04.880550 3336 raft_consensus.cc:2258] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:04.881845 2646 tablet_service.cc:1514] Processing DeleteTablet for tablet a4277045ec344ad7a48db9b14bea5e7b with delete_type TABLET_DATA_DELETED (replaced by tablet 0a2f98dd905d40959dc676f94a4a595b) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:04.882143 3336 ts_tablet_manager.cc:1905] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:04.882678 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet a4277045ec344ad7a48db9b14bea5e7b with delete_type TABLET_DATA_DELETED (replaced by tablet 0a2f98dd905d40959dc676f94a4a595b) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:04.885809 3438 raft_consensus.cc:482] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:04.886205 3438 raft_consensus.cc:504] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.887885 3438 leader_election.cc:288] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:04.892844 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet a4277045ec344ad7a48db9b14bea5e7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4277045ec344ad7a48db9b14bea5e7b already in progress: creating tablet
I20260927 23:28:04.894069 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
I20260927 23:28:04.895160 2386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 04f670db808c4e289c655679da9d2d9d
I20260927 23:28:04.892271 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1b6b75dad24c609a10693728ba8e20" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:04.895761 2972 raft_consensus.cc:2454] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:04.896672 2386 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 128) took 1751 ms. Trace:
I20260927 23:28:04.897477 2608 leader_election.cc:302] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:04.898177 3250 raft_consensus.cc:2781] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:04.898471 3250 raft_consensus.cc:482] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:04.898728 3250 raft_consensus.cc:3037] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:04.896936 2386 rpcz_store.cc:276] 0927 23:28:03.144925 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.145434 (+ 509us) service_pool.cc:227] Handling call
0927 23:28:03.152411 (+ 6977us) spinlock_profiling.cc:243] Waited 6.23 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b3466 00007f8bb3bf4b34 00007f8bb3bf4ac2 00007f8bb3bf49f1 00007f8bb3bf496c 00007f8bb3bf3c22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:03.153418 (+ 1007us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:03.153915 (+ 497us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:28:03.154055 (+ 140us) catalog_manager.cc:2761] Locking tablets
0927 23:28:04.281764 (+1127709us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:28:04.645759 (+363995us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:28:04.645972 (+ 213us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:04.894943 (+248971us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:04.896613 (+ 1670us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:04.283852 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:04.284190 (+ 338us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:04.284214 (+ 24us) write_op.cc:435] Acquired schema lock
0927 23:28:04.284225 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:04.284547 (+ 322us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:04.284593 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:04.284614 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:04.284888 (+ 274us) tablet.cc:699] Row locks acquired
0927 23:28:04.284903 (+ 15us) write_op.cc:260] PREPARE: finished
0927 23:28:04.285006 (+ 103us) write_op.cc:270] Start()
0927 23:28:04.285084 (+ 78us) write_op.cc:276] Timestamp: P: 1790551684284929 usec, L: 0
0927 23:28:04.285101 (+ 17us) op_driver.cc:331] REPLICATION: starting
0927 23:28:04.377812 (+ 92711us) log.cc:842] Serialized 753 byte log entry
0927 23:28:04.379446 (+ 1634us) spinlock_profiling.cc:243] Waited 92.1 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068471 00007f8bb40df198 00007f8bb4540ade 00007f8bb45402b8 00007f8bb4544ba1 00007f8bb4544b59 00007f8bb4544ae9 00007f8bb4544ab1 00007f8bb4543dad 0000000000580784 00000000005805b9
0927 23:28:04.640454 (+261008us) op_driver.cc:444] REPLICATION: finished
0927 23:28:04.641617 (+ 1163us) write_op.cc:301] APPLY: starting
0927 23:28:04.641680 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:04.641799 (+ 119us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:04.641823 (+ 24us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:04.642089 (+ 266us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:04.642240 (+ 151us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:04.642287 (+ 47us) write_op.cc:312] APPLY: finished
0927 23:28:04.642502 (+ 215us) log.cc:842] Serialized 29 byte log entry
0927 23:28:04.642683 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:04.642755 (+ 72us) write_op.cc:454] Released schema lock
0927 23:28:04.642789 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:04.642989 (+ 200us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:04.643014 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":14325504,"child_traces":[["op",{"apply.queue_time_us":975,"apply.run_cpu_time_us":1608,"apply.run_wall_time_us":1608,"num_ops":2,"prepare.queue_time_us":108,"prepare.run_cpu_time_us":4419,"prepare.run_wall_time_us":100543,"replication_time_us":355261,"spinlock_wait_cycles":211881728}]]}
I20260927 23:28:04.901518 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 04f670db808c4e289c655679da9d2d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:04.903457 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_511501de200744a59414f1dd8bf6b2fa"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:04.904999 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 04f670db808c4e289c655679da9d2d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:04.905267 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_511501de200744a59414f1dd8bf6b2fa in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:04.906250 3311 raft_consensus.cc:482] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:28:04.906492 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 04f670db808c4e289c655679da9d2d9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04f670db808c4e289c655679da9d2d9d already in progress: creating tablet
I20260927 23:28:04.906648 3311 raft_consensus.cc:504] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.908423 3311 leader_election.cc:288] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:04.910027 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7767a32c07454fe6a92154429a3b6899" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:04.910233 3250 raft_consensus.cc:504] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:04.911123 2907 leader_election.cc:341] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:04.915580 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1b6b75dad24c609a10693728ba8e20" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:04.916324 2972 raft_consensus.cc:3037] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:04.917483 3072 tablet_bootstrap.cc:492] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:04.916855 2385 catalog_manager.cc:5495] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:04.917902 3072 ts_tablet_manager.cc:1397] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.124s user 0.010s sys 0.002s
I20260927 23:28:04.918455 2385 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:04.919931 3072 raft_consensus.cc:348] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.908803 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1b6b75dad24c609a10693728ba8e20" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:04.919976 2385 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1d299ca52cfc493499b5eaf0acdeb5f9 (table default.Table_bf18dd7dd7984680bf41384772c15e61 [id=1af28dc40a8d408ba31c695e8563efc2]). Expected version 1 got 0
I20260927 23:28:04.923249 2972 raft_consensus.cc:2454] T ef1b6b75dad24c609a10693728ba8e20 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:04.924227 2608 leader_election.cc:302] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:04.924952 3438 raft_consensus.cc:2781] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
W20260927 23:28:04.925818 2609 leader_election.cc:341] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:04.927781 3167 tablet_bootstrap.cc:492] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:04.928155 3167 ts_tablet_manager.cc:1397] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.358s user 0.012s sys 0.004s
I20260927 23:28:04.929041 2805 tablet_service.cc:1514] Processing DeleteTablet for tablet a4277045ec344ad7a48db9b14bea5e7b with delete_type TABLET_DATA_DELETED (replaced by tablet 0a2f98dd905d40959dc676f94a4a595b) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:04.929807 3250 leader_election.cc:288] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:04.921006 3072 raft_consensus.cc:374] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.935662 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7767a32c07454fe6a92154429a3b6899" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:04.936111 3072 raft_consensus.cc:729] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.936635 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet 04f670db808c4e289c655679da9d2d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:04.936745 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 1da38499af164d7fb5c1a8d65167ad24 (DEFAULT_TABLE table=default.Table_92a19ba603ef488c972786364cf49c70 [id=b12aae65f933476ab57974d4b328a266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.936723 3072 consensus_queue.cc:260] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:04.937800 2906 leader_election.cc:341] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:04.938159 2906 leader_election.cc:302] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:04.940541 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1b6b75dad24c609a10693728ba8e20" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:04.941941 2609 leader_election.cc:341] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:04.943750 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet a4277045ec344ad7a48db9b14bea5e7b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a4277045ec344ad7a48db9b14bea5e7b already in progress: creating tablet
W20260927 23:28:04.944407 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 04f670db808c4e289c655679da9d2d9d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 04f670db808c4e289c655679da9d2d9d already in progress: creating tablet
I20260927 23:28:04.944869 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1da38499af164d7fb5c1a8d65167ad24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.930747 2385 catalog_manager.cc:5495] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:04.945798 2385 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
I20260927 23:28:04.947435 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (DEFAULT_TABLE table=default.Table_d2b79535214c487fb59255b089a1b5bd [id=47c1f33b77964104b1be5f25b788d8af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.948381 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.949750 3368 raft_consensus.cc:2726] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:04.951327 3072 ts_tablet_manager.cc:1428] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.033s user 0.005s sys 0.000s
I20260927 23:28:04.952630 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 1da38499af164d7fb5c1a8d65167ad24 (DEFAULT_TABLE table=default.Table_92a19ba603ef488c972786364cf49c70 [id=b12aae65f933476ab57974d4b328a266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.956900 3167 raft_consensus.cc:348] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.957553 3167 raft_consensus.cc:374] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.957515 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (DEFAULT_TABLE table=default.Table_d2b79535214c487fb59255b089a1b5bd [id=47c1f33b77964104b1be5f25b788d8af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:04.957772 3167 raft_consensus.cc:729] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.958307 3167 consensus_queue.cc:260] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:04.958575 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.960579 3167 ts_tablet_manager.cc:1428] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.032s user 0.005s sys 0.000s
I20260927 23:28:04.960878 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1da38499af164d7fb5c1a8d65167ad24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:04.961308 3167 tablet_bootstrap.cc:492] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:04.963598 3072 tablet_bootstrap.cc:492] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:04.967278 3167 tablet_bootstrap.cc:654] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.977145 3070 tablet_bootstrap.cc:492] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:04.977551 3070 ts_tablet_manager.cc:1397] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.238s user 0.017s sys 0.032s
I20260927 23:28:04.979625 3070 raft_consensus.cc:348] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.980314 3070 raft_consensus.cc:374] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:04.980594 3070 raft_consensus.cc:729] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:04.982681 3070 consensus_queue.cc:260] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.984802 3250 consensus_queue.cc:1035] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:04.986726 3070 ts_tablet_manager.cc:1428] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260927 23:28:04.987063 3438 raft_consensus.cc:686] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:04.988603 3438 consensus_queue.cc:237] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:04.987545 3070 tablet_bootstrap.cc:492] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:04.992415 2822 raft_consensus.cc:3037] T 1d299ca52cfc493499b5eaf0acdeb5f9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:04.995455 3167 tablet_bootstrap.cc:492] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:04.995908 3167 ts_tablet_manager.cc:1397] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.035s user 0.009s sys 0.001s
I20260927 23:28:04.998484 3473 tablet.cc:1722] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:05.002713 3070 tablet_bootstrap.cc:654] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:04.998528 3167 raft_consensus.cc:348] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.005761 3167 raft_consensus.cc:374] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.006065 3167 raft_consensus.cc:729] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
W20260927 23:28:05.012686 2906 consensus_peers.cc:487] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 95216fdc2e104a2e9fc4bc424bdbe5e9. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:05.015326 3167 consensus_queue.cc:260] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.024566 3072 tablet_bootstrap.cc:654] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.026243 3167 ts_tablet_manager.cc:1428] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.030s user 0.001s sys 0.004s
I20260927 23:28:05.030330 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (DEFAULT_TABLE table=default.Table_d2b79535214c487fb59255b089a1b5bd [id=47c1f33b77964104b1be5f25b788d8af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:05.033743 3336 ts_tablet_manager.cc:1918] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:05.034085 3336 log.cc:1196] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/66f2842218fc42e5b1b83314e5f1a158
I20260927 23:28:05.034097 2803 tablet_service.cc:1467] Processing CreateTablet for tablet 2411d7c53a154fafad9edda9fa75bb36 (DEFAULT_TABLE table=default.Table_b967cf863a6848919c6542cb6bbb1c70 [id=597f1d374b754602850d1dbc3bd4e913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:05.035082 3167 tablet_bootstrap.cc:492] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:05.035338 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:05.036643 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2411d7c53a154fafad9edda9fa75bb36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:05.040637 3167 tablet_bootstrap.cc:654] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.041802 3438 raft_consensus.cc:482] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.042347 3438 raft_consensus.cc:504] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.045059 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e104d49543934e5db3b188e1db065cdc" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:05.045644 2969 raft_consensus.cc:2454] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:05.046759 2608 leader_election.cc:302] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:05.049301 3481 tablet.cc:1722] T 1d299ca52cfc493499b5eaf0acdeb5f9 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:05.050220 3250 raft_consensus.cc:2781] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:05.050626 3250 raft_consensus.cc:482] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:05.050982 3250 raft_consensus.cc:3037] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.051873 3438 leader_election.cc:288] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:05.054277 3480 tablet.cc:1722] T 1d299ca52cfc493499b5eaf0acdeb5f9 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:05.062304 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e104d49543934e5db3b188e1db065cdc" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:05.063024 2824 raft_consensus.cc:2454] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:05.067334 3336 ts_tablet_manager.cc:1939] T 66f2842218fc42e5b1b83314e5f1a158 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:05.071693 3070 tablet_bootstrap.cc:492] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:05.072109 3070 ts_tablet_manager.cc:1397] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.085s user 0.009s sys 0.000s
I20260927 23:28:05.074139 3070 raft_consensus.cc:348] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.074769 3070 raft_consensus.cc:374] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.075026 3070 raft_consensus.cc:729] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
W20260927 23:28:05.091372 2609 consensus_peers.cc:487] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:05.091540 2393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 2d837daee7a04b338813519acf53dfae consensus state has no leader
I20260927 23:28:05.092787 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_e10895f6ce7143a4b328dfe1818e5c64"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:05.094136 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e10895f6ce7143a4b328dfe1818e5c64 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:05.097493 3070 consensus_queue.cc:260] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.101378 3438 consensus_queue.cc:1035] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:05.108273 3072 tablet_bootstrap.cc:492] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:05.109284 3072 ts_tablet_manager.cc:1397] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.146s user 0.008s sys 0.003s
W20260927 23:28:05.110635 2608 consensus_peers.cc:487] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Couldn't send request to peer de216a4a8e4b4e1b857d8647a3b05f6d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a80bff324902498fa5c04fe8cbc05f9d. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:05.110175 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 80) took 1944 ms. Trace:
I20260927 23:28:05.113098 3336 rpcz_store.cc:276] 0927 23:28:03.165622 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.202514 (+ 36892us) service_pool.cc:227] Handling call
0927 23:28:04.875017 (+1672503us) ts_tablet_manager.cc:1145] Deleting tablet 66f2842218fc42e5b1b83314e5f1a158
0927 23:28:04.875054 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:04.984013 (+108959us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:05.033526 (+ 49513us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:05.110123 (+ 76597us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1671227}
I20260927 23:28:05.117700 3070 ts_tablet_manager.cc:1428] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.045s user 0.005s sys 0.000s
I20260927 23:28:05.118556 3070 tablet_bootstrap.cc:492] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:05.118673 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 66f2842218fc42e5b1b83314e5f1a158 (table default.Table_33057639607344a0bcd2f4850a382324 [id=6b1d660c32a74492bb7b47fea2e1132d]) successfully deleted
I20260927 23:28:05.120142 3336 tablet_replica.cc:330] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:05.122571 3336 raft_consensus.cc:2229] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:05.125059 3336 raft_consensus.cc:2258] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:05.116925 3072 raft_consensus.cc:348] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.130237 3072 raft_consensus.cc:374] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.130594 3072 raft_consensus.cc:729] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:05.131655 3072 consensus_queue.cc:260] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.137444 3438 raft_consensus.cc:482] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.144152 3368 raft_consensus.cc:482] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.144690 3368 raft_consensus.cc:504] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.146678 3438 raft_consensus.cc:504] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.148443 3438 leader_election.cc:288] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:05.148469 3368 leader_election.cc:288] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:05.149740 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "933d4c631a1345f9a117b87d0e669117" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:05.129729 3336 ts_tablet_manager.cc:1905] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:05.150089 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae1e85c32a94f868ef48ad471462b3e" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:05.151196 2906 leader_election.cc:341] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:05.155447 2609 leader_election.cc:341] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.159287 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "933d4c631a1345f9a117b87d0e669117" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:05.159873 2972 raft_consensus.cc:2454] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:05.160926 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae1e85c32a94f868ef48ad471462b3e" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:05.162213 2608 leader_election.cc:302] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:05.163341 3438 raft_consensus.cc:2781] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:05.163707 3438 raft_consensus.cc:482] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:05.164065 3438 raft_consensus.cc:3037] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:05.165555 2907 leader_election.cc:341] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.165932 2907 leader_election.cc:302] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:05.167987 3368 raft_consensus.cc:2726] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:05.178416 3070 tablet_bootstrap.cc:654] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.183084 3167 tablet_bootstrap.cc:492] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:05.183597 3167 ts_tablet_manager.cc:1397] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.149s user 0.012s sys 0.000s
I20260927 23:28:05.185657 3167 raft_consensus.cc:348] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:05.186262 3167 raft_consensus.cc:374] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.186475 3167 raft_consensus.cc:729] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:05.187028 3167 consensus_queue.cc:260] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:05.188823 3167 ts_tablet_manager.cc:1428] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 23:28:05.175905 3438 raft_consensus.cc:504] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.192214 3438 leader_election.cc:288] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:05.192447 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "933d4c631a1345f9a117b87d0e669117" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:05.193562 2609 leader_election.cc:341] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.195858 2387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet ac19099dd7bf4b669d504a654ee69482
I20260927 23:28:05.197113 2387 catalog_manager.cc:6565] ReplaceTablet: tablet ac19099dd7bf4b669d504a654ee69482 deleted and replaced by tablet 9eb27269a590476f878ee52a2aefd141
I20260927 23:28:05.200714 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet ac19099dd7bf4b669d504a654ee69482 with delete_type TABLET_DATA_DELETED (replaced by tablet 9eb27269a590476f878ee52a2aefd141) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:05.202127 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet ac19099dd7bf4b669d504a654ee69482 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac19099dd7bf4b669d504a654ee69482 already in progress: creating tablet
I20260927 23:28:05.203720 2803 tablet_service.cc:1514] Processing DeleteTablet for tablet ac19099dd7bf4b669d504a654ee69482 with delete_type TABLET_DATA_DELETED (replaced by tablet 9eb27269a590476f878ee52a2aefd141) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:05.205791 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_86ee4c8bf9ad4144bbddded60848863f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:05.207692 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86ee4c8bf9ad4144bbddded60848863f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:05.209223 3070 tablet_bootstrap.cc:492] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:05.209602 3070 ts_tablet_manager.cc:1397] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.091s user 0.006s sys 0.012s
I20260927 23:28:05.209852 2388 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet e973ddc4efcd42f7a1f71ec7caac5e66
I20260927 23:28:05.210690 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_b5538a7a7f4b4d49bad39084c724c90b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:05.212000 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5538a7a7f4b4d49bad39084c724c90b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:05.211735 3070 raft_consensus.cc:348] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.212322 3070 raft_consensus.cc:374] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.212656 3070 raft_consensus.cc:729] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:05.213177 3070 consensus_queue.cc:260] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:05.214315 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet ac19099dd7bf4b669d504a654ee69482 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac19099dd7bf4b669d504a654ee69482 already in progress: creating tablet
I20260927 23:28:05.220541 2386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_a2512a58a2fe47ff928e54a9b0a8f0f1" } modify_external_catalogs: true
I20260927 23:28:05.221159 2386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_a2512a58a2fe47ff928e54a9b0a8f0f1" } modify_external_catalogs: true
I20260927 23:28:05.223392 3070 ts_tablet_manager.cc:1428] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s
I20260927 23:28:05.231179 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 2411d7c53a154fafad9edda9fa75bb36 (DEFAULT_TABLE table=default.Table_b967cf863a6848919c6542cb6bbb1c70 [id=597f1d374b754602850d1dbc3bd4e913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:05.232367 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2411d7c53a154fafad9edda9fa75bb36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:05.170725 3072 ts_tablet_manager.cc:1428] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.060s user 0.005s sys 0.000s
I20260927 23:28:05.239274 3167 tablet_bootstrap.cc:492] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:05.239261 3072 tablet_bootstrap.cc:492] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:05.243742 3438 raft_consensus.cc:482] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.244194 3438 raft_consensus.cc:504] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.246038 3438 leader_election.cc:288] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:05.247932 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d837daee7a04b338813519acf53dfae" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:05.250420 2609 leader_election.cc:341] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:05.252360 2377 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 2d837daee7a04b338813519acf53dfae (table default.Table_08f6c5348e2b41feba84b48a70ac7a8c [id=5e610846ba4f4cd7907236b723747067], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2d837daee7a04b338813519acf53dfae consensus state has no leader
I20260927 23:28:05.254025 3167 tablet_bootstrap.cc:654] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.264376 3070 tablet_bootstrap.cc:492] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:05.264456 2803 tablet_service.cc:1467] Processing CreateTablet for tablet 40bce5362d2541329b253716bbd99e07 (DEFAULT_TABLE table=default.Table_576d409caeb746d48f1b67bdc96437bc [id=9dec5171c0de4e9cb35f2c21342f4e6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:05.265574 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40bce5362d2541329b253716bbd99e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:05.276387 3070 tablet_bootstrap.cc:654] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.280066 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet ac19099dd7bf4b669d504a654ee69482 with delete_type TABLET_DATA_DELETED (replaced by tablet 9eb27269a590476f878ee52a2aefd141) from {username='slave'} at 127.0.0.1:44952
W20260927 23:28:05.279389 3489 log.cc:924] Time spent T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d: Append to log took a long time: real 0.156s user 0.000s sys 0.001s
I20260927 23:28:05.286150 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "933d4c631a1345f9a117b87d0e669117" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:05.286738 2972 raft_consensus.cc:3037] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.292065 3070 tablet_bootstrap.cc:492] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:05.292527 3070 ts_tablet_manager.cc:1397] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.028s user 0.012s sys 0.007s
I20260927 23:28:05.294720 3070 raft_consensus.cc:348] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.295408 3070 raft_consensus.cc:374] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.295681 3070 raft_consensus.cc:729] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:05.296423 3070 consensus_queue.cc:260] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.298657 3070 ts_tablet_manager.cc:1428] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260927 23:28:05.299467 3070 tablet_bootstrap.cc:492] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:05.301728 3167 tablet_bootstrap.cc:492] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:05.302105 3167 ts_tablet_manager.cc:1397] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.063s user 0.008s sys 0.000s
I20260927 23:28:05.302484 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 40bce5362d2541329b253716bbd99e07 (DEFAULT_TABLE table=default.Table_576d409caeb746d48f1b67bdc96437bc [id=9dec5171c0de4e9cb35f2c21342f4e6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:05.303392 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40bce5362d2541329b253716bbd99e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:05.304268 3070 tablet_bootstrap.cc:654] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.304191 3167 raft_consensus.cc:348] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.304828 3167 raft_consensus.cc:374] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.305073 3167 raft_consensus.cc:729] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:05.305684 3167 consensus_queue.cc:260] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.309762 3167 ts_tablet_manager.cc:1428] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260927 23:28:05.311548 3167 tablet_bootstrap.cc:492] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:05.315092 3070 tablet_bootstrap.cc:492] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:05.315496 3070 ts_tablet_manager.cc:1397] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260927 23:28:05.316512 3167 tablet_bootstrap.cc:654] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.317800 3070 raft_consensus.cc:348] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.318536 3070 raft_consensus.cc:374] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.318805 3070 raft_consensus.cc:729] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:05.319506 3070 consensus_queue.cc:260] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.322453 3070 ts_tablet_manager.cc:1428] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260927 23:28:05.323261 3070 tablet_bootstrap.cc:492] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:05.330101 2967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d837daee7a04b338813519acf53dfae" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:05.330705 2967 raft_consensus.cc:2454] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:05.334160 2608 leader_election.cc:302] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:05.335135 3438 raft_consensus.cc:2781] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:05.335428 3438 raft_consensus.cc:482] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:05.335705 3438 raft_consensus.cc:3037] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.336963 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 2411d7c53a154fafad9edda9fa75bb36 (DEFAULT_TABLE table=default.Table_b967cf863a6848919c6542cb6bbb1c70 [id=597f1d374b754602850d1dbc3bd4e913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:05.337900 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2411d7c53a154fafad9edda9fa75bb36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:05.342365 3167 tablet_bootstrap.cc:492] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:05.342734 3167 ts_tablet_manager.cc:1397] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.031s user 0.003s sys 0.005s
I20260927 23:28:05.344864 3167 raft_consensus.cc:348] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.345522 3167 raft_consensus.cc:374] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.345747 3167 raft_consensus.cc:729] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:05.347563 3438 raft_consensus.cc:504] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.352674 3167 consensus_queue.cc:260] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.353470 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d837daee7a04b338813519acf53dfae" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:05.355208 3167 ts_tablet_manager.cc:1428] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260927 23:28:05.355989 3167 tablet_bootstrap.cc:492] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
W20260927 23:28:05.356112 2609 leader_election.cc:341] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.360977 3438 leader_election.cc:288] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:05.368819 3167 tablet_bootstrap.cc:654] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.370587 2393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_f298fb86e27d466eaee4ae90137687d0" } new_table_name: "default.Table_4dd942a54bfc41a882f7e66a38e4d1fb" modify_external_catalogs: true
I20260927 23:28:05.328863 3368 raft_consensus.cc:482] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.378257 3368 raft_consensus.cc:504] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:05.381107 2609 consensus_peers.cc:487] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:05.383121 3250 raft_consensus.cc:504] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.391788 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 40bce5362d2541329b253716bbd99e07 (DEFAULT_TABLE table=default.Table_576d409caeb746d48f1b67bdc96437bc [id=9dec5171c0de4e9cb35f2c21342f4e6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:05.392961 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40bce5362d2541329b253716bbd99e07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:05.392083 2384 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet cc1c0da6ea5c4a09a71555f37becae33 (table default.Table_29404653bfbb4e5d8192bcb604084df0 [id=baad95258e1a4358a58abc3fc1cc3737]). Expected version 1 got 0
I20260927 23:28:05.394496 3167 tablet_bootstrap.cc:492] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:05.394455 2384 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:05.394867 3167 ts_tablet_manager.cc:1397] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.000s
I20260927 23:28:05.395342 2384 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1d299ca52cfc493499b5eaf0acdeb5f9 (table default.Table_bf18dd7dd7984680bf41384772c15e61 [id=1af28dc40a8d408ba31c695e8563efc2]). Expected version 1 got 0
I20260927 23:28:05.396785 2392 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:51972 (request call id 140) took 1596 ms. Trace:
I20260927 23:28:05.397015 3167 raft_consensus.cc:348] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.397615 3167 raft_consensus.cc:374] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.397714 3368 leader_election.cc:288] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:05.397835 3167 raft_consensus.cc:729] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:05.398134 3368 raft_consensus.cc:482] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.398478 3368 raft_consensus.cc:504] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.397111 2392 rpcz_store.cc:276] 0927 23:28:03.799976 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.802221 (+ 2245us) service_pool.cc:227] Handling call
0927 23:28:03.803250 (+ 1029us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:03.806892 (+ 3642us) catalog_manager.cc:3616] Apply alter schema
0927 23:28:03.807482 (+ 590us) catalog_manager.cc:3639] Acquired catalog manager lock
0927 23:28:03.807694 (+ 212us) catalog_manager.cc:3786] Updating metadata on disk
0927 23:28:04.046771 (+239077us) catalog_manager.cc:3817] Committing alterations to in-memory state
0927 23:28:05.396715 (+1349944us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:03.930711 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:03.931059 (+ 348us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:03.931081 (+ 22us) write_op.cc:435] Acquired schema lock
0927 23:28:03.931092 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:03.935279 (+ 4187us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:03.935322 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:03.935346 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:03.935504 (+ 158us) tablet.cc:699] Row locks acquired
0927 23:28:03.935517 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:28:03.935625 (+ 108us) write_op.cc:270] Start()
0927 23:28:03.935705 (+ 80us) write_op.cc:276] Timestamp: P: 1790551683916979 usec, L: 0
0927 23:28:03.935720 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:03.936190 (+ 470us) log.cc:842] Serialized 405 byte log entry
0927 23:28:04.040901 (+104711us) op_driver.cc:444] REPLICATION: finished
0927 23:28:04.044736 (+ 3835us) write_op.cc:301] APPLY: starting
0927 23:28:04.044794 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:04.045409 (+ 615us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:04.045509 (+ 100us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:04.045716 (+ 207us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:04.045857 (+ 141us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:04.045896 (+ 39us) write_op.cc:312] APPLY: finished
0927 23:28:04.046077 (+ 181us) log.cc:842] Serialized 23 byte log entry
0927 23:28:04.046230 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:04.046280 (+ 50us) write_op.cc:454] Released schema lock
0927 23:28:04.046311 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:04.046464 (+ 153us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:04.046489 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":82,"apply.run_cpu_time_us":1902,"apply.run_wall_time_us":1902,"num_ops":1,"prepare.queue_time_us":121850,"prepare.run_cpu_time_us":1806,"prepare.run_wall_time_us":24830,"replication_time_us":88821}]]}
W20260927 23:28:05.403986 2389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 30) took 1762 ms (client timeout 1000 ms). Trace:
I20260927 23:28:05.404596 3167 consensus_queue.cc:260] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.400113 3368 leader_election.cc:288] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:05.410838 3167 ts_tablet_manager.cc:1428] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.016s user 0.005s sys 0.001s
I20260927 23:28:05.411595 3167 tablet_bootstrap.cc:492] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:05.412067 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e with delete_type TABLET_DATA_DELETED (replaced by tablet f58349edf0db40f98951e116fc2582b0) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:05.412673 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet 7096e434ac9f4290ad58ba4f7e7c2513 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:05.413743 3341 tablet_replica.cc:330] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:05.414371 3341 raft_consensus.cc:2229] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:05.414845 3341 raft_consensus.cc:2258] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:05.421036 3341 ts_tablet_manager.cc:1905] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:05.427105 2389 rpcz_store.cc:269] 0927 23:28:03.641260 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.651539 (+ 10279us) service_pool.cc:227] Handling call
0927 23:28:05.403901 (+1752362us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":4}
W20260927 23:28:05.419066 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e because the tablet was not found. No further retry: Not found: Tablet not found: 3acdfe7faa1a4d5d8e340f09f1cae93e
I20260927 23:28:05.428999 2972 raft_consensus.cc:2454] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
W20260927 23:28:05.429466 2391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 24) took 1748 ms (client timeout 1000 ms). Trace:
W20260927 23:28:05.429811 2391 rpcz_store.cc:269] 0927 23:28:03.681208 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.681333 (+ 125us) service_pool.cc:227] Handling call
0927 23:28:05.429411 (+1748078us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:04.499985 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:04.500372 (+ 387us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:04.500406 (+ 34us) write_op.cc:435] Acquired schema lock
0927 23:28:04.500420 (+ 14us) tablet.cc:660] Decoding operations
0927 23:28:04.500677 (+ 257us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:04.500714 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:04.500736 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:04.500886 (+ 150us) tablet.cc:699] Row locks acquired
0927 23:28:04.500900 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:04.501005 (+ 105us) write_op.cc:270] Start()
0927 23:28:04.501088 (+ 83us) write_op.cc:276] Timestamp: P: 1790551684500906 usec, L: 0
0927 23:28:04.501103 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:04.501571 (+ 468us) log.cc:842] Serialized 524 byte log entry
0927 23:28:04.724905 (+223334us) op_driver.cc:444] REPLICATION: finished
0927 23:28:04.725903 (+ 998us) write_op.cc:301] APPLY: starting
0927 23:28:04.725964 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:04.726069 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:04.726177 (+ 108us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:04.726355 (+ 178us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:04.726489 (+ 134us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:04.726534 (+ 45us) write_op.cc:312] APPLY: finished
0927 23:28:04.726720 (+ 186us) log.cc:842] Serialized 23 byte log entry
0927 23:28:04.726868 (+ 148us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:04.726928 (+ 60us) write_op.cc:454] Released schema lock
0927 23:28:04.726959 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:04.727132 (+ 173us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:04.727157 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":840,"apply.run_cpu_time_us":1411,"apply.run_wall_time_us":1410,"num_ops":1,"prepare.queue_time_us":1627,"prepare.run_cpu_time_us":1822,"prepare.run_wall_time_us":24181,"replication_time_us":223729}]]}
I20260927 23:28:05.444410 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_ad787a15ff9743029d703111e0a20bdf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:05.446007 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ad787a15ff9743029d703111e0a20bdf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:05.449496 2392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_db0f806a353a4dbdb9d64635383bc826" } modify_external_catalogs: true
I20260927 23:28:05.450827 2392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_db0f806a353a4dbdb9d64635383bc826" } modify_external_catalogs: true
I20260927 23:28:05.461910 3072 tablet_bootstrap.cc:654] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.467885 3488 tablet.cc:1722] T 1d299ca52cfc493499b5eaf0acdeb5f9 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:05.471036 3341 ts_tablet_manager.cc:1918] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:05.405830 3070 tablet_bootstrap.cc:654] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.416489 2652 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cce47b050b445dfba7be5ea3b6a8636 with delete_type TABLET_DATA_DELETED (replaced by tablet 1da38499af164d7fb5c1a8d65167ad24) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:05.422940 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 410294c28c49404e9c547b27111712eb with delete_type TABLET_DATA_DELETED (replaced by tablet 8cd0972f635d4979abf5fd30e9b02b1d) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:05.477933 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 9cce47b050b445dfba7be5ea3b6a8636 because the tablet was not found. No further retry: Not found: Tablet not found: 9cce47b050b445dfba7be5ea3b6a8636
W20260927 23:28:05.478646 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 410294c28c49404e9c547b27111712eb because the tablet was not found. No further retry: Not found: Tablet not found: 410294c28c49404e9c547b27111712eb
I20260927 23:28:05.437558 3438 raft_consensus.cc:482] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.479470 3438 raft_consensus.cc:504] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.484570 3438 leader_election.cc:288] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:05.438882 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e with delete_type TABLET_DATA_DELETED (replaced by tablet f58349edf0db40f98951e116fc2582b0) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:05.449935 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47c2f22abbe40e7ba8ea945ab96aeaf" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:05.460965 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4277045ec344ad7a48db9b14bea5e7b" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:05.485446 2672 raft_consensus.cc:2454] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:05.468572 3486 raft_consensus.cc:482] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.486140 3486 raft_consensus.cc:504] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:05.487123 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e because the tablet was not found. No further retry: Not found: Tablet not found: 3acdfe7faa1a4d5d8e340f09f1cae93e
I20260927 23:28:05.487954 3486 leader_election.cc:288] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:05.501945 3497 raft_consensus.cc:482] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.502442 3497 raft_consensus.cc:504] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.503804 3480 tablet.cc:1722] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:05.509801 3341 log.cc:1196] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/7096e434ac9f4290ad58ba4f7e7c2513
I20260927 23:28:05.512036 2391 master_service.cc:946] ReplaceTablet: received request to replace tablet ac19099dd7bf4b669d504a654ee69482 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:05.519791 3341 ts_tablet_manager.cc:1939] T 7096e434ac9f4290ad58ba4f7e7c2513 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:05.484812 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d37dba66b84d05abf675d3eaf39a30" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:05.521533 2823 raft_consensus.cc:2454] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:05.521797 2608 leader_election.cc:302] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:05.522373 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47c2f22abbe40e7ba8ea945ab96aeaf" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:05.489429 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7767a32c07454fe6a92154429a3b6899" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:05.508651 3481 tablet.cc:1722] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:05.523492 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 7096e434ac9f4290ad58ba4f7e7c2513 (table default.Table_3963531cde4641e785245322c0a23fab [id=1cac6a4a98734d4abf99d93891e1f077]) successfully deleted
I20260927 23:28:05.525198 2609 leader_election.cc:302] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:05.472047 3167 tablet_bootstrap.cc:654] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.520922 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4277045ec344ad7a48db9b14bea5e7b" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:05.529004 3486 raft_consensus.cc:2781] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:05.529479 3486 raft_consensus.cc:686] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:05.529587 3250 leader_election.cc:288] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:28:05.530474 2609 leader_election.cc:341] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.531240 3438 raft_consensus.cc:2781] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:05.531538 3438 raft_consensus.cc:482] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:05.531824 3438 raft_consensus.cc:3037] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.533226 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e104d49543934e5db3b188e1db065cdc" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:05.533766 2821 raft_consensus.cc:3037] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.536489 3486 consensus_queue.cc:237] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
W20260927 23:28:05.541833 2378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260927 23:28:05.544529 3070 tablet_bootstrap.cc:492] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:05.545130 3070 ts_tablet_manager.cc:1397] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.222s user 0.009s sys 0.002s
I20260927 23:28:05.547313 3070 raft_consensus.cc:348] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.552966 2391 master_service.cc:946] ReplaceTablet: received request to replace tablet 702ebdeb7c5a4b7d900dbb00236cb8ae from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:05.555807 3368 raft_consensus.cc:482] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.556377 3368 raft_consensus.cc:504] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.558284 3368 leader_election.cc:288] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:05.561311 3505 raft_consensus.cc:482] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.561808 3505 raft_consensus.cc:504] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.555137 2821 raft_consensus.cc:2454] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:05.567830 3505 leader_election.cc:288] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:05.578025 3072 tablet_bootstrap.cc:492] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:05.578549 3072 ts_tablet_manager.cc:1397] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.339s user 0.006s sys 0.007s
I20260927 23:28:05.579154 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_9f0dec29a5d74ddaa48ef255c54b22cd"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:05.580885 3072 raft_consensus.cc:348] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.581477 3072 raft_consensus.cc:374] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.581707 3072 raft_consensus.cc:729] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
W20260927 23:28:05.584707 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9f0dec29a5d74ddaa48ef255c54b22cd in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:05.590258 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d37dba66b84d05abf675d3eaf39a30" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:05.590991 2821 raft_consensus.cc:2454] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:05.504220 3497 leader_election.cc:288] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:05.594651 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d837daee7a04b338813519acf53dfae" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:05.595260 2969 raft_consensus.cc:3037] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:05.598237 2378 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.60:41571, user_credentials={real_user=slave}} blocked reactor thread for 56408.1us
I20260927 23:28:05.602746 2969 raft_consensus.cc:2454] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:05.550130 3438 raft_consensus.cc:504] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.606166 3070 raft_consensus.cc:374] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.606484 3070 raft_consensus.cc:729] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:05.607643 3438 leader_election.cc:288] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:05.565454 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "933d4c631a1345f9a117b87d0e669117" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:05.568755 2609 leader_election.cc:302] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:05.611222 3438 raft_consensus.cc:2781] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:05.611334 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d37dba66b84d05abf675d3eaf39a30" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:05.611689 3438 raft_consensus.cc:686] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:05.612036 2821 raft_consensus.cc:3037] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.612803 3438 consensus_queue.cc:237] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.615501 3070 consensus_queue.cc:260] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.624130 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbbe17f54d1418b95efc3943db0b6c1" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:05.630366 2386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 74fc2f03baae4f70b38b8079deb8a854
W20260927 23:28:05.632975 2393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet a8df6d0652634def9c8946c346dea702 has no consensus state
I20260927 23:28:05.639195 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 74fc2f03baae4f70b38b8079deb8a854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:05 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:05.639993 3335 tablet_replica.cc:330] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:05.640820 3335 raft_consensus.cc:2229] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:05.648664 2386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_57fe58bb7f964bdf9bbfcd03dd684f66" } new_table_name: "default.Table_e007bebc730046b48b94b18f382b2cb7" modify_external_catalogs: true
I20260927 23:28:05.652582 2803 tablet_service.cc:1514] Processing DeleteTablet for tablet 74fc2f03baae4f70b38b8079deb8a854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:05 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:05.653337 3341 tablet_replica.cc:330] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:05.654047 3341 raft_consensus.cc:2229] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:05.654999 3341 raft_consensus.cc:2258] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:05.656991 3341 ts_tablet_manager.cc:1905] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:05.669602 3335 raft_consensus.cc:2258] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:05.670639 3167 tablet_bootstrap.cc:492] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:05.671089 3167 ts_tablet_manager.cc:1397] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.260s user 0.027s sys 0.017s
I20260927 23:28:05.671921 3335 ts_tablet_manager.cc:1905] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:05.673213 3167 raft_consensus.cc:348] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:05.673847 3167 raft_consensus.cc:374] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.674101 3167 raft_consensus.cc:729] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:05.674649 3167 consensus_queue.cc:260] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:05.675179 2821 raft_consensus.cc:2454] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:05.605473 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d37dba66b84d05abf675d3eaf39a30" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:05.679049 2972 raft_consensus.cc:2454] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:05.606721 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7767a32c07454fe6a92154429a3b6899" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:05.679795 2971 raft_consensus.cc:2454] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:05.607733 2967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e104d49543934e5db3b188e1db065cdc" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:05.680478 2967 raft_consensus.cc:3037] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.686899 3167 ts_tablet_manager.cc:1428] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260927 23:28:05.688876 2967 raft_consensus.cc:2454] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
W20260927 23:28:05.618148 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 134591us
I20260927 23:28:05.691982 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbbe17f54d1418b95efc3943db0b6c1" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:05.622869 3336 ts_tablet_manager.cc:1918] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:05.692938 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d37dba66b84d05abf675d3eaf39a30" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:05.693284 3336 log.cc:1196] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/5efbd0211cf3408cb5ea9640d8da6694
W20260927 23:28:05.637634 2906 leader_election.cc:341] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.654810 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "933d4c631a1345f9a117b87d0e669117" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:05.693960 2670 raft_consensus.cc:2379] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:05.694191 2968 raft_consensus.cc:2379] T 933d4c631a1345f9a117b87d0e669117 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
W20260927 23:28:05.694933 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 57341.3us
W20260927 23:28:05.695456 2906 leader_election.cc:341] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.695969 2906 leader_election.cc:302] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:05.696599 3506 raft_consensus.cc:2781] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:05.696894 3506 raft_consensus.cc:482] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:05.697163 3506 raft_consensus.cc:3037] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.697275 3167 tablet_bootstrap.cc:492] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:05.700023 3070 ts_tablet_manager.cc:1428] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.155s user 0.005s sys 0.000s
W20260927 23:28:05.700663 2906 leader_election.cc:341] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.700986 3070 tablet_bootstrap.cc:492] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:05.705397 2608 leader_election.cc:302] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:05.706745 2907 leader_election.cc:341] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.708745 3336 ts_tablet_manager.cc:1939] T 5efbd0211cf3408cb5ea9640d8da6694 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:05.710582 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 81) took 2544 ms. Trace:
I20260927 23:28:05.710879 3336 rpcz_store.cc:276] 0927 23:28:03.166017 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.205741 (+ 39724us) service_pool.cc:227] Handling call
0927 23:28:05.119991 (+1914250us) ts_tablet_manager.cc:1145] Deleting tablet 5efbd0211cf3408cb5ea9640d8da6694
0927 23:28:05.120033 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:05.412870 (+292837us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:05.622646 (+209776us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:05.710520 (+ 87874us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1913370}
I20260927 23:28:05.712934 3336 tablet_replica.cc:330] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:05.713698 3336 raft_consensus.cc:2229] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:05.714772 3336 raft_consensus.cc:2258] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:05.678546 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 74fc2f03baae4f70b38b8079deb8a854 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:05 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:05.663906 3072 consensus_queue.cc:260] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.714217 2907 leader_election.cc:302] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:05.718191 2761 leader_election.cc:302] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:05.718926 3454 raft_consensus.cc:3037] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.720182 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 5efbd0211cf3408cb5ea9640d8da6694 (table default.Table_7c86d4308c504c4ba092bb6634326544 [id=0aa0174883974249b1120d0f02be108d]) successfully deleted
I20260927 23:28:05.723855 3501 raft_consensus.cc:2726] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:05.723062 3488 tablet.cc:1722] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:05.725545 2907 leader_election.cc:302] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:05.726233 3501 raft_consensus.cc:2781] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:05.726611 3501 raft_consensus.cc:482] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:05.727125 3501 raft_consensus.cc:3037] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.729996 3167 tablet_bootstrap.cc:654] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.737588 3072 ts_tablet_manager.cc:1428] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.159s user 0.005s sys 0.000s
I20260927 23:28:05.738509 3072 tablet_bootstrap.cc:492] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:05.739374 3335 ts_tablet_manager.cc:1918] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:05.739786 3335 log.cc:1196] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/74fc2f03baae4f70b38b8079deb8a854
I20260927 23:28:05.747745 3070 tablet_bootstrap.cc:654] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:05.748754 3454 raft_consensus.cc:2726] T 933d4c631a1345f9a117b87d0e669117 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:05.749845 2608 leader_election.cc:302] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:05.751111 3250 raft_consensus.cc:2781] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:05.751503 3250 raft_consensus.cc:482] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:05.751886 3250 raft_consensus.cc:3037] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.756583 3341 ts_tablet_manager.cc:1918] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:05.757275 3438 raft_consensus.cc:2781] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:05.758652 3438 raft_consensus.cc:686] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:05.761257 2609 leader_election.cc:302] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:05.762068 3486 raft_consensus.cc:2781] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:05.761427 3167 tablet_bootstrap.cc:492] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:05.764129 3167 ts_tablet_manager.cc:1397] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.067s user 0.009s sys 0.003s
I20260927 23:28:05.766767 3167 raft_consensus.cc:348] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.767508 3167 raft_consensus.cc:374] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:05.767761 3167 raft_consensus.cc:729] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:05.767844 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d37dba66b84d05abf675d3eaf39a30" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:05.768388 3167 consensus_queue.cc:260] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.760466 3438 consensus_queue.cc:237] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.770838 3341 log.cc:1196] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/74fc2f03baae4f70b38b8079deb8a854
W20260927 23:28:05.776350 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 50884.8us
I20260927 23:28:05.777021 3341 ts_tablet_manager.cc:1939] T 74fc2f03baae4f70b38b8079deb8a854 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
W20260927 23:28:05.777109 2907 leader_election.cc:341] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.777592 2907 leader_election.cc:302] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:05.778306 3496 raft_consensus.cc:2726] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:05.772244 3167 ts_tablet_manager.cc:1428] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260927 23:28:05.781181 3486 raft_consensus.cc:686] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:05.783504 3486 consensus_queue.cc:237] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:05.785163 3335 ts_tablet_manager.cc:1939] T 74fc2f03baae4f70b38b8079deb8a854 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:05.811654 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 74fc2f03baae4f70b38b8079deb8a854 (table default.Table_a2512a58a2fe47ff928e54a9b0a8f0f1 [id=052cb2b2fbda4be3825be24c2325b85c]) successfully deleted
I20260927 23:28:05.819664 3501 raft_consensus.cc:504] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.821822 3501 leader_election.cc:288] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:05.822960 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4277045ec344ad7a48db9b14bea5e7b" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:05.823350 3506 raft_consensus.cc:504] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
W20260927 23:28:05.824483 2906 leader_election.cc:341] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.832288 3167 tablet_bootstrap.cc:492] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:05.833452 2969 raft_consensus.cc:2379] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5ce671f028a34ea4b8f0520c2689c9a1 in current term 1: Already voted for candidate de216a4a8e4b4e1b857d8647a3b05f6d in this term.
I20260927 23:28:05.843330 3454 raft_consensus.cc:482] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.843873 3454 raft_consensus.cc:504] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:05.845840 3454 leader_election.cc:288] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:05.847144 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 74fc2f03baae4f70b38b8079deb8a854 (table default.Table_a2512a58a2fe47ff928e54a9b0a8f0f1 [id=052cb2b2fbda4be3825be24c2325b85c]) successfully deleted
I20260927 23:28:05.847999 3336 ts_tablet_manager.cc:1905] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:05.858928 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a80bff324902498fa5c04fe8cbc05f9d" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:05.861357 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_4e3a8231cd1a426f82b9fecd2dc11400"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:05.862813 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4e3a8231cd1a426f82b9fecd2dc11400 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:05.863677 3519 raft_consensus.cc:482] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.864146 3519 raft_consensus.cc:504] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.867509 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884b80a1d31349c98f75525416b3c527" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:05.868788 2609 leader_election.cc:341] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.866168 3519 leader_election.cc:288] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:05.853266 3486 consensus_queue.cc:1035] T fac49c5b79e347158621dda52f79e912 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260927 23:28:05.873039 3167 tablet_bootstrap.cc:654] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:05.879132 2609 consensus_peers.cc:487] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:05.874158 3250 raft_consensus.cc:504] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.884640 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7767a32c07454fe6a92154429a3b6899" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:05.885208 2821 raft_consensus.cc:3037] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.886006 2391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 702ebdeb7c5a4b7d900dbb00236cb8ae
I20260927 23:28:05.887177 2391 catalog_manager.cc:6565] ReplaceTablet: tablet 702ebdeb7c5a4b7d900dbb00236cb8ae deleted and replaced by tablet 8fadaafd49b04b9483a6a50f2f7d5be7
I20260927 23:28:05.888233 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae with delete_type TABLET_DATA_DELETED (replaced by tablet 8fadaafd49b04b9483a6a50f2f7d5be7) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:05.890772 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 702ebdeb7c5a4b7d900dbb00236cb8ae already in progress: creating tablet
I20260927 23:28:05.893117 2821 raft_consensus.cc:2454] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:05.894269 2803 tablet_service.cc:1514] Processing DeleteTablet for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae with delete_type TABLET_DATA_DELETED (replaced by tablet 8fadaafd49b04b9483a6a50f2f7d5be7) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:05.894317 2609 leader_election.cc:302] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:05.895257 3250 leader_election.cc:288] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:05.895804 3250 raft_consensus.cc:2781] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:05.896387 3250 raft_consensus.cc:686] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:05.897815 3250 consensus_queue.cc:237] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:05.901587 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 702ebdeb7c5a4b7d900dbb00236cb8ae already in progress: creating tablet
I20260927 23:28:05.911959 2967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7767a32c07454fe6a92154429a3b6899" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:05.912564 2967 raft_consensus.cc:3037] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.918696 3454 raft_consensus.cc:482] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.919329 3454 raft_consensus.cc:504] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:05.923274 2967 raft_consensus.cc:2454] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
W20260927 23:28:05.924782 2763 leader_election.cc:341] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 5efbd0211cf3408cb5ea9640d8da6694
I20260927 23:28:05.924944 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884b80a1d31349c98f75525416b3c527" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:05.925437 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_b349caab5cc84842819f9a9bbfece301"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:05.925858 3454 leader_election.cc:288] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:05.925683 2969 raft_consensus.cc:2454] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
W20260927 23:28:05.926870 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b349caab5cc84842819f9a9bbfece301 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:05.928040 3519 raft_consensus.cc:482] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:05.928555 3519 raft_consensus.cc:504] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.930538 3519 leader_election.cc:288] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:05.931202 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47c2f22abbe40e7ba8ea945ab96aeaf" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:05.932471 2609 leader_election.cc:341] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:05.929939 3506 leader_election.cc:288] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:05.951731 3072 tablet_bootstrap.cc:654] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:05.955096 2386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 5c87d497da884544920b3be9c78daccf has no consensus state
I20260927 23:28:05.955566 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4277045ec344ad7a48db9b14bea5e7b" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:05.956158 2673 raft_consensus.cc:3037] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:05.956502 2386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_f1571426f866470d94cfe827bd30ba79" } modify_external_catalogs: true
I20260927 23:28:05.957039 2386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_f1571426f866470d94cfe827bd30ba79" } modify_external_catalogs: true
I20260927 23:28:05.965157 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae with delete_type TABLET_DATA_DELETED (replaced by tablet 8fadaafd49b04b9483a6a50f2f7d5be7) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:05.965590 2673 raft_consensus.cc:2454] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:05.956175 3486 consensus_queue.cc:1035] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
W20260927 23:28:05.978812 2761 leader_election.cc:341] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: a80bff324902498fa5c04fe8cbc05f9d
I20260927 23:28:05.979212 2761 leader_election.cc:302] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:05.984069 3454 raft_consensus.cc:2726] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:05.984972 3070 tablet_bootstrap.cc:492] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:05.985395 3070 ts_tablet_manager.cc:1397] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.285s user 0.009s sys 0.000s
I20260927 23:28:05.987665 2387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_576d409caeb746d48f1b67bdc96437bc" } new_table_name: "default.Table_82bad979f0da4702af4e135570772d7e" modify_external_catalogs: true
W20260927 23:28:05.988498 2761 leader_election.cc:341] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: 5efbd0211cf3408cb5ea9640d8da6694
I20260927 23:28:05.989029 2761 leader_election.cc:302] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:05.989698 3454 raft_consensus.cc:2726] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:05.996409 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47c2f22abbe40e7ba8ea945ab96aeaf" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:05.997161 2969 raft_consensus.cc:2454] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:06.014746 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d37dba66b84d05abf675d3eaf39a30" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:06.015641 2821 raft_consensus.cc:2379] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:06.021121 3486 raft_consensus.cc:482] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:06.021317 2391 master_service.cc:946] ReplaceTablet: received request to replace tablet 2d837daee7a04b338813519acf53dfae from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:06.021641 3486 raft_consensus.cc:504] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.023829 3486 leader_election.cc:288] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:06.026585 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04f670db808c4e289c655679da9d2d9d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:06.028474 2609 leader_election.cc:341] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:06.029350 3167 tablet_bootstrap.cc:492] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:06.029124 3070 raft_consensus.cc:348] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.029752 3167 ts_tablet_manager.cc:1397] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.198s user 0.008s sys 0.000s
I20260927 23:28:06.029793 3070 raft_consensus.cc:374] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:06.030035 3070 raft_consensus.cc:729] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:06.030766 3070 consensus_queue.cc:260] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:05.955770 2608 leader_election.cc:302] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:06.032056 3167 raft_consensus.cc:348] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.032703 3167 raft_consensus.cc:374] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:06.032933 3167 raft_consensus.cc:729] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:06.038441 3072 tablet_bootstrap.cc:492] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:06.038937 3072 ts_tablet_manager.cc:1397] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.301s user 0.007s sys 0.003s
I20260927 23:28:06.040992 3070 ts_tablet_manager.cc:1428] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.055s user 0.006s sys 0.000s
I20260927 23:28:06.041357 3072 raft_consensus.cc:348] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.041855 3070 tablet_bootstrap.cc:492] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:06.042038 3072 raft_consensus.cc:374] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:06.042279 3072 raft_consensus.cc:729] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:06.043144 3486 raft_consensus.cc:2781] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:06.043537 3486 raft_consensus.cc:482] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:06.043816 3486 raft_consensus.cc:3037] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:06.053452 2386 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 8fadaafd49b04b9483a6a50f2f7d5be7
I20260927 23:28:06.042909 3072 consensus_queue.cc:260] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.063375 3072 ts_tablet_manager.cc:1428] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.024s user 0.003s sys 0.004s
I20260927 23:28:06.063716 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_70bf888024144eb0b5afd48ee484fb17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:06.064419 3072 tablet_bootstrap.cc:492] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
W20260927 23:28:06.065136 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_70bf888024144eb0b5afd48ee484fb17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:06.067063 3486 raft_consensus.cc:504] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.033531 3167 consensus_queue.cc:260] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.069710 3486 leader_election.cc:288] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:06.069711 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884b80a1d31349c98f75525416b3c527" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:06.070329 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 114617us
I20260927 23:28:06.071264 2907 leader_election.cc:302] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:06.073041 3501 raft_consensus.cc:2781] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:06.073486 2821 raft_consensus.cc:3037] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:06.073526 3501 raft_consensus.cc:686] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:06.074247 3501 consensus_queue.cc:237] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.075284 3167 ts_tablet_manager.cc:1428] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.045s user 0.005s sys 0.000s
I20260927 23:28:06.076114 3167 tablet_bootstrap.cc:492] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:06.082257 2821 raft_consensus.cc:2454] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:06.089699 2609 leader_election.cc:302] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:06.093518 3072 tablet_bootstrap.cc:654] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:06.098392 2967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04f670db808c4e289c655679da9d2d9d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:06.099066 2967 raft_consensus.cc:2454] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
W20260927 23:28:06.099903 2387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 40bce5362d2541329b253716bbd99e07 consensus state has no leader
I20260927 23:28:06.106308 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "884b80a1d31349c98f75525416b3c527" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:06.106931 2969 raft_consensus.cc:3037] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:06.107928 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_b30d329785704a9eaf36610d3a96a7a9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:06.109365 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b30d329785704a9eaf36610d3a96a7a9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:06.119330 3486 raft_consensus.cc:2781] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:06.120400 3486 raft_consensus.cc:686] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:06.121287 3486 consensus_queue.cc:237] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.126579 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "84d37dba66b84d05abf675d3eaf39a30" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:06.166355 2608 leader_election.cc:302] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:06.167464 3516 raft_consensus.cc:2781] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:06.167783 3516 raft_consensus.cc:482] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:06.168082 3516 raft_consensus.cc:3037] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:06.176277 3516 raft_consensus.cc:504] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.177945 2386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_d2b79535214c487fb59255b089a1b5bd" } new_table_name: "default.Table_777042bde16f4fc4a33505d169021646" modify_external_catalogs: true
I20260927 23:28:06.179071 3516 leader_election.cc:288] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:06.179051 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47c2f22abbe40e7ba8ea945ab96aeaf" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:06.180858 2609 leader_election.cc:341] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:06.188865 3167 tablet_bootstrap.cc:654] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:06.193058 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_a368d1f8ac2544ba9c5b637c76f92176"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:06.209164 2386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 consensus state has no leader
I20260927 23:28:06.192536 2821 raft_consensus.cc:3037] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:06.223203 2969 raft_consensus.cc:2454] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:06.225689 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_da4f2d8a1534460a8d060ea730c5e3e8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:06.233702 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 66412us
I20260927 23:28:06.234802 2608 leader_election.cc:302] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:06.246886 3070 tablet_bootstrap.cc:654] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:06.250278 3336 ts_tablet_manager.cc:1918] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:06.250921 3336 log.cc:1196] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/9cce47b050b445dfba7be5ea3b6a8636
I20260927 23:28:06.251487 3336 ts_tablet_manager.cc:1939] T 9cce47b050b445dfba7be5ea3b6a8636 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
W20260927 23:28:06.253332 2377 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet a8df6d0652634def9c8946c346dea702 (table default.Table_4dd942a54bfc41a882f7e66a38e4d1fb [id=996f8a130c7f48b4b23b74dad8f5fee4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet a8df6d0652634def9c8946c346dea702 has no consensus state
I20260927 23:28:06.255995 3519 raft_consensus.cc:2781] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:06.256367 3519 raft_consensus.cc:482] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:06.256668 3519 raft_consensus.cc:3037] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:06.259968 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a368d1f8ac2544ba9c5b637c76f92176 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:06.267577 3167 tablet_bootstrap.cc:492] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:06.268101 3167 ts_tablet_manager.cc:1397] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.192s user 0.008s sys 0.001s
W20260927 23:28:06.272084 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_da4f2d8a1534460a8d060ea730c5e3e8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:06.289312 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 54581us
I20260927 23:28:06.291451 2907 leader_election.cc:302] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:06.293140 3506 raft_consensus.cc:2726] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260927 23:28:06.295408 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47c2f22abbe40e7ba8ea945ab96aeaf" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:06.296137 2968 raft_consensus.cc:3037] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:06.308082 3454 raft_consensus.cc:482] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:06.271950 3519 raft_consensus.cc:504] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.308583 3454 raft_consensus.cc:504] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:06.310352 3454 leader_election.cc:288] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:06.319106 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d837daee7a04b338813519acf53dfae" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:06.320190 2969 raft_consensus.cc:2379] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:06.321683 3501 raft_consensus.cc:482] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:06.322157 3501 raft_consensus.cc:504] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.325042 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac19099dd7bf4b669d504a654ee69482" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:06.326515 2906 leader_election.cc:341] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:06.328804 2386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_b30d329785704a9eaf36610d3a96a7a9" } modify_external_catalogs: true
I20260927 23:28:06.329439 2386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_b30d329785704a9eaf36610d3a96a7a9" } modify_external_catalogs: true
I20260927 23:28:06.332823 3167 raft_consensus.cc:348] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.333573 3167 raft_consensus.cc:374] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:06.333863 3167 raft_consensus.cc:729] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:06.334769 3167 consensus_queue.cc:260] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.336193 3519 leader_election.cc:288] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:06.337764 2387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_2ec8e4445f1d425ea0a7dae9f9d6200b" } new_table_name: "default.Table_96276d84ee11493d89908e4ff16dc7fe" modify_external_catalogs: true
I20260927 23:28:06.342051 3501 leader_election.cc:288] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:06.343904 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04f670db808c4e289c655679da9d2d9d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:06.351064 2609 leader_election.cc:341] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:06.352519 2821 raft_consensus.cc:3037] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:06.364090 3486 raft_consensus.cc:482] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:06.366168 3486 raft_consensus.cc:504] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.368211 3486 leader_election.cc:288] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:06.369692 3167 ts_tablet_manager.cc:1428] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.101s user 0.005s sys 0.001s
I20260927 23:28:06.377970 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae1e85c32a94f868ef48ad471462b3e" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:06.386795 2609 leader_election.cc:341] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:06.388876 3167 tablet_bootstrap.cc:492] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:06.391927 3506 raft_consensus.cc:482] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:06.392359 3506 raft_consensus.cc:504] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.394073 3506 leader_election.cc:288] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:06.395967 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f71f948891b4d7fa95c3a1fbd92ce1e" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:06.396836 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac19099dd7bf4b669d504a654ee69482" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:06.398118 2906 leader_election.cc:341] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:06.402050 3072 tablet_bootstrap.cc:492] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:06.402577 3072 ts_tablet_manager.cc:1397] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.338s user 0.032s sys 0.041s
W20260927 23:28:06.417198 2907 leader_election.cc:341] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:06.417711 2907 leader_election.cc:302] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:06.417869 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f71f948891b4d7fa95c3a1fbd92ce1e" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:06.418977 3506 raft_consensus.cc:2726] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:06.420166 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04f670db808c4e289c655679da9d2d9d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
W20260927 23:28:06.420948 2907 leader_election.cc:341] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:06.421096 2971 raft_consensus.cc:3037] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:06.421366 2907 leader_election.cc:302] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:06.422525 3506 raft_consensus.cc:2726] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:06.424817 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae1e85c32a94f868ef48ad471462b3e" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:06.425506 2972 raft_consensus.cc:2454] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:06.431502 2968 raft_consensus.cc:2454] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
W20260927 23:28:06.435289 2387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet cfbbe17f54d1418b95efc3943db0b6c1 consensus state has no leader
I20260927 23:28:06.437331 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_8a68a0857d4f40a99f630c8ac395c419"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:06.439247 2386 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet e32cb382eedc4e389638312502460578
I20260927 23:28:06.441783 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_8abcb4595c95442ba71077feec547d2c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:06.437084 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 86) took 3269 ms. Trace:
I20260927 23:28:06.452201 3336 rpcz_store.cc:276] 0927 23:28:03.167762 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.216523 (+ 48761us) service_pool.cc:227] Handling call
0927 23:28:05.712806 (+2496283us) ts_tablet_manager.cc:1145] Deleting tablet 9cce47b050b445dfba7be5ea3b6a8636
0927 23:28:05.712841 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:05.949665 (+236824us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:06.250073 (+300408us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:06.436995 (+186922us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2495355}
I20260927 23:28:06.453495 3336 tablet_replica.cc:330] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:06.454231 3336 raft_consensus.cc:2229] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 2 LEADER]: Raft consensus shutting down.
I20260927 23:28:06.455106 3336 pending_rounds.cc:62] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d: Trying to abort 1 pending ops.
I20260927 23:28:06.455331 3336 pending_rounds.cc:68] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d: Aborting op as it isn't in flight: id { term: 2 index: 1 } timestamp: 7334099696929779712 op_type: NO_OP noop_request { }
I20260927 23:28:06.455672 3336 raft_consensus.cc:2866] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:28:06.455958 3336 raft_consensus.cc:2258] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:06.443671 2971 raft_consensus.cc:2454] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:06.457392 3072 raft_consensus.cc:348] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.459367 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d837daee7a04b338813519acf53dfae" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:06.458030 3072 raft_consensus.cc:374] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:06.460783 3072 raft_consensus.cc:729] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:06.461360 2763 leader_election.cc:302] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:06.464766 3072 consensus_queue.cc:260] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.467497 3336 ts_tablet_manager.cc:1905] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:06.453559 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 9cce47b050b445dfba7be5ea3b6a8636 (table default.Table_92a19ba603ef488c972786364cf49c70 [id=b12aae65f933476ab57974d4b328a266]) successfully deleted
W20260927 23:28:06.469296 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8a68a0857d4f40a99f630c8ac395c419 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:06.470351 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8abcb4595c95442ba71077feec547d2c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:06.473119 3547 raft_consensus.cc:3037] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:06.477793 3072 ts_tablet_manager.cc:1428] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.075s user 0.005s sys 0.003s
I20260927 23:28:06.478736 3072 tablet_bootstrap.cc:492] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:06.492395 3167 tablet_bootstrap.cc:654] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:06.494707 2608 leader_election.cc:302] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:06.495584 3536 raft_consensus.cc:2781] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:06.495903 3536 raft_consensus.cc:482] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:06.496194 3536 raft_consensus.cc:3037] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:06.500383 3547 raft_consensus.cc:2726] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:06.503481 3536 raft_consensus.cc:504] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.508253 2608 leader_election.cc:302] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:06.515537 3519 raft_consensus.cc:2781] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:06.516000 3519 raft_consensus.cc:686] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:06.516855 3519 consensus_queue.cc:237] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.517802 3072 tablet_bootstrap.cc:654] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:06.520741 2608 leader_election.cc:302] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:06.521588 2387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_10714592f65b41f5ad17642613dc4537" } new_table_name: "default.Table_d609f64deb9647f79c581917277e1f93" modify_external_catalogs: true
I20260927 23:28:06.521798 3539 raft_consensus.cc:2781] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:06.530968 3454 raft_consensus.cc:482] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5ce671f028a34ea4b8f0520c2689c9a1)
I20260927 23:28:06.531448 3454 raft_consensus.cc:504] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:06.532058 3070 tablet_bootstrap.cc:492] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:06.532449 3070 ts_tablet_manager.cc:1397] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.491s user 0.009s sys 0.004s
I20260927 23:28:06.535553 3454 leader_election.cc:288] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:06.542516 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac49c5b79e347158621dda52f79e912" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:06.545240 2967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fac49c5b79e347158621dda52f79e912" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:06.546597 2761 leader_election.cc:302] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:06.547605 3454 raft_consensus.cc:2726] T fac49c5b79e347158621dda52f79e912 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:06.550149 3536 leader_election.cc:288] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:06.553908 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae1e85c32a94f868ef48ad471462b3e" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:06.560014 2967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae1e85c32a94f868ef48ad471462b3e" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:06.560691 2967 raft_consensus.cc:3037] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:06.564365 3072 tablet_bootstrap.cc:492] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:06.564833 3072 ts_tablet_manager.cc:1397] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.086s user 0.010s sys 0.000s
I20260927 23:28:06.550832 3539 raft_consensus.cc:686] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:06.567183 3072 raft_consensus.cc:348] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.567808 3072 raft_consensus.cc:374] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:06.567404 3539 consensus_queue.cc:237] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.568023 3072 raft_consensus.cc:729] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:06.571172 3167 tablet_bootstrap.cc:492] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:06.571570 3167 ts_tablet_manager.cc:1397] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.183s user 0.008s sys 0.000s
I20260927 23:28:06.573591 3167 raft_consensus.cc:348] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.574213 3167 raft_consensus.cc:374] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:06.574430 3167 raft_consensus.cc:729] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
W20260927 23:28:06.555298 2609 leader_election.cc:341] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:06.579721 3540 raft_consensus.cc:482] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:06.580165 3540 raft_consensus.cc:504] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.581980 3540 leader_election.cc:288] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:06.556572 3250 consensus_queue.cc:1035] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:06.561200 3070 raft_consensus.cc:348] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.585232 3070 raft_consensus.cc:374] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:06.585562 3070 raft_consensus.cc:729] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:06.586175 3070 consensus_queue.cc:260] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.587657 3336 ts_tablet_manager.cc:1918] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.1
I20260927 23:28:06.588048 3336 log.cc:1196] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/95216fdc2e104a2e9fc4bc424bdbe5e9
I20260927 23:28:06.588348 3070 ts_tablet_manager.cc:1428] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.056s user 0.004s sys 0.002s
I20260927 23:28:06.588666 3072 consensus_queue.cc:260] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.589229 3070 tablet_bootstrap.cc:492] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:06.595604 3070 tablet_bootstrap.cc:654] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:06.595655 3072 ts_tablet_manager.cc:1428] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.031s user 0.006s sys 0.000s
I20260927 23:28:06.596560 3072 tablet_bootstrap.cc:492] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:06.602713 3540 consensus_queue.cc:1035] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:06.613791 2386 master_service.cc:946] ReplaceTablet: received request to replace tablet cc1c0da6ea5c4a09a71555f37becae33 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:06.617049 3336 ts_tablet_manager.cc:1939] T 95216fdc2e104a2e9fc4bc424bdbe5e9 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
W20260927 23:28:06.617223 2387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 38be781ead0e470e9928881594b4b98f has no consensus state
I20260927 23:28:06.619740 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_0b85d1f76d2b4cd68123d6b031a5fa72"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:06.620177 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 87) took 2867 ms. Trace:
I20260927 23:28:06.620513 3336 rpcz_store.cc:276] 0927 23:28:03.752910 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.775497 (+ 22587us) service_pool.cc:227] Handling call
0927 23:28:06.453326 (+2677829us) ts_tablet_manager.cc:1145] Deleting tablet 95216fdc2e104a2e9fc4bc424bdbe5e9
0927 23:28:06.453373 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:06.539170 (+ 85797us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:06.587411 (+ 48241us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:06.620120 (+ 32709us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2677235}
I20260927 23:28:06.623253 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 (table default.Table_877402b3ef234b9c99bbdab0842b3885 [id=29be7f6208c242568f403c0389282d93]) successfully deleted
I20260927 23:28:06.628434 3070 tablet_bootstrap.cc:492] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:06.628896 3070 ts_tablet_manager.cc:1397] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.040s user 0.011s sys 0.001s
I20260927 23:28:06.631073 3070 raft_consensus.cc:348] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.631738 3070 raft_consensus.cc:374] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:06.631987 3070 raft_consensus.cc:729] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:06.631834 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbbe17f54d1418b95efc3943db0b6c1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:06.632650 3070 consensus_queue.cc:260] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.634987 3070 ts_tablet_manager.cc:1428] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260927 23:28:06.635810 3070 tablet_bootstrap.cc:492] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
W20260927 23:28:06.637050 2609 leader_election.cc:341] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:06.640837 3336 tablet_replica.cc:330] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:06.641557 3336 raft_consensus.cc:2229] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:06.641960 3336 raft_consensus.cc:2258] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:06.643536 3336 ts_tablet_manager.cc:1905] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:06.649626 3070 tablet_bootstrap.cc:654] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:06.653357 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbbe17f54d1418b95efc3943db0b6c1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:06.653831 3072 tablet_bootstrap.cc:654] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:06.654039 2971 raft_consensus.cc:2454] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:06.661139 2386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet cc1c0da6ea5c4a09a71555f37becae33
I20260927 23:28:06.662889 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet cc1c0da6ea5c4a09a71555f37becae33 with delete_type TABLET_DATA_DELETED (replaced by tablet 78e099028f7847d1b3de7d437be97de0) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:06.664163 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet cc1c0da6ea5c4a09a71555f37becae33 with delete_type TABLET_DATA_DELETED (replaced by tablet 78e099028f7847d1b3de7d437be97de0) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:06.664391 2803 tablet_service.cc:1514] Processing DeleteTablet for tablet cc1c0da6ea5c4a09a71555f37becae33 with delete_type TABLET_DATA_DELETED (replaced by tablet 78e099028f7847d1b3de7d437be97de0) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:06.665807 3539 raft_consensus.cc:482] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:06.666409 3539 raft_consensus.cc:504] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.675691 3070 tablet_bootstrap.cc:492] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:06.676229 3070 ts_tablet_manager.cc:1397] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.041s user 0.006s sys 0.003s
I20260927 23:28:06.678470 3070 raft_consensus.cc:348] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.679112 3070 raft_consensus.cc:374] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:06.679358 3070 raft_consensus.cc:729] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:06.680066 3072 tablet_bootstrap.cc:492] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:06.679912 3070 consensus_queue.cc:260] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.681866 3070 ts_tablet_manager.cc:1428] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260927 23:28:06.682783 3070 tablet_bootstrap.cc:492] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:06.684514 3558 tablet_replica.cc:330] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:06.684927 2386 catalog_manager.cc:6565] ReplaceTablet: tablet cc1c0da6ea5c4a09a71555f37becae33 deleted and replaced by tablet 78e099028f7847d1b3de7d437be97de0
I20260927 23:28:06.687345 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_79ed7e54234b47fc8f068e605aa6121a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:06.689131 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_79ed7e54234b47fc8f068e605aa6121a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:06.689196 3536 consensus_queue.cc:1035] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260927 23:28:06.679013 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0b85d1f76d2b4cd68123d6b031a5fa72 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:06.690492 3072 ts_tablet_manager.cc:1397] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.094s user 0.010s sys 0.000s
I20260927 23:28:06.693499 3072 raft_consensus.cc:348] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.694172 3072 raft_consensus.cc:374] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:06.694389 3072 raft_consensus.cc:729] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:06.694944 3072 consensus_queue.cc:260] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.696878 3072 ts_tablet_manager.cc:1428] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260927 23:28:06.697615 3072 tablet_bootstrap.cc:492] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:06.702651 3072 tablet_bootstrap.cc:654] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:06.690197 3070 tablet_bootstrap.cc:654] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:06.727272 3558 raft_consensus.cc:2229] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:06.728164 3558 raft_consensus.cc:2258] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:06.679898 3167 consensus_queue.cc:260] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.731201 3167 ts_tablet_manager.cc:1428] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.159s user 0.007s sys 0.000s
I20260927 23:28:06.733318 3539 leader_election.cc:288] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:06.734174 3167 tablet_bootstrap.cc:492] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:06.738408 3559 tablet_replica.cc:330] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:06.739498 3559 raft_consensus.cc:2229] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:06.717540 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f71f948891b4d7fa95c3a1fbd92ce1e" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:06.741515 3516 consensus_queue.cc:1035] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:06.742835 3516 consensus_queue.cc:1035] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:06.744604 2608 leader_election.cc:302] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:06.745559 2609 leader_election.cc:341] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:06.746104 3516 raft_consensus.cc:2781] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:06.746419 3516 raft_consensus.cc:482] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:06.746702 3516 raft_consensus.cc:3037] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:06.758914 3454 raft_consensus.cc:482] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:06.759414 3454 raft_consensus.cc:504] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.761538 3454 leader_election.cc:288] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:06.763032 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1b6b75dad24c609a10693728ba8e20" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:06.765771 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ef1b6b75dad24c609a10693728ba8e20" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:06.767457 2761 leader_election.cc:302] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:06.768240 3454 raft_consensus.cc:2726] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:06.774317 3336 ts_tablet_manager.cc:1918] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:06.779059 3336 log.cc:1196] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/7767a32c07454fe6a92154429a3b6899
I20260927 23:28:06.780359 3336 ts_tablet_manager.cc:1939] T 7767a32c07454fe6a92154429a3b6899 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:06.789554 3539 consensus_queue.cc:1035] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:06.804654 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 88) took 2979 ms. Trace:
W20260927 23:28:06.805470 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 61169.9us
I20260927 23:28:06.807729 3336 rpcz_store.cc:276] 0927 23:28:03.825195 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.825755 (+ 560us) service_pool.cc:227] Handling call
0927 23:28:06.640684 (+2814929us) ts_tablet_manager.cc:1145] Deleting tablet 7767a32c07454fe6a92154429a3b6899
0927 23:28:06.640724 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:06.699720 (+ 58996us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:06.774095 (+ 74375us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:06.804583 (+ 30488us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2814401}
I20260927 23:28:06.806635 3516 raft_consensus.cc:504] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.814057 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:06.814786 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
I20260927 23:28:06.817422 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 2d837daee7a04b338813519acf53dfae (table default.Table_08f6c5348e2b41feba84b48a70ac7a8c [id=5e610846ba4f4cd7907236b723747067]). Expected version 1 got 0
I20260927 23:28:06.817094 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbbe17f54d1418b95efc3943db0b6c1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:06.818508 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f71f948891b4d7fa95c3a1fbd92ce1e" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:06.820897 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 7767a32c07454fe6a92154429a3b6899 (table default.Table_6568f80896124508811607305bf18ff9 [id=adbf504950b64a08996f102f4ac2d662]) successfully deleted
I20260927 23:28:06.820938 2971 raft_consensus.cc:2454] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:06.829030 3559 raft_consensus.cc:2258] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:06.842523 2391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 2d837daee7a04b338813519acf53dfae
I20260927 23:28:06.846678 2391 catalog_manager.cc:6565] ReplaceTablet: tablet 2d837daee7a04b338813519acf53dfae deleted and replaced by tablet 9af9b87abf1f4f78b3140a91f2525066
I20260927 23:28:06.846771 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d837daee7a04b338813519acf53dfae with delete_type TABLET_DATA_DELETED (replaced by tablet 9af9b87abf1f4f78b3140a91f2525066) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:06.846307 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d837daee7a04b338813519acf53dfae with delete_type TABLET_DATA_DELETED (replaced by tablet 9af9b87abf1f4f78b3140a91f2525066) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:06.848212 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 90) took 2937 ms. Trace:
I20260927 23:28:06.848590 3336 rpcz_store.cc:276] 0927 23:28:03.910570 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.914461 (+ 3891us) service_pool.cc:227] Handling call
0927 23:28:06.847947 (+2933486us) ts_tablet_manager.cc:1145] Deleting tablet 2572268ca47346b0b78abc6a9ca7f908
0927 23:28:06.847992 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:06.848159 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2920278}
W20260927 23:28:06.849800 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 2572268ca47346b0b78abc6a9ca7f908 because the tablet was not found. No further retry: Not found: Tablet not found: 2572268ca47346b0b78abc6a9ca7f908
I20260927 23:28:06.853453 2803 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d837daee7a04b338813519acf53dfae with delete_type TABLET_DATA_DELETED (replaced by tablet 9af9b87abf1f4f78b3140a91f2525066) from {username='slave'} at 127.0.0.1:42492
W20260927 23:28:06.869850 2609 leader_election.cc:341] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:06.880476 2967 raft_consensus.cc:2454] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:06.883770 3516 leader_election.cc:288] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:06.918404 3070 tablet_bootstrap.cc:492] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:06.918895 3070 ts_tablet_manager.cc:1397] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.236s user 0.005s sys 0.004s
I20260927 23:28:06.768018 3558 ts_tablet_manager.cc:1905] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:06.925869 3070 raft_consensus.cc:348] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.926748 3070 raft_consensus.cc:374] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:06.926994 3070 raft_consensus.cc:729] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:06.927686 3070 consensus_queue.cc:260] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:06.937956 3336 tablet_replica.cc:330] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:06.939190 2388 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:06.939893 2388 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 6ca42b47d8044adc99a82859bf79e606 (table default.Table_2ca15ccb4f1f4b388143d9810dbb4ec6 [id=29d80c3f3f21428b819d211d3e10f621]). Expected version 2 got 1
I20260927 23:28:06.941385 2388 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 1d299ca52cfc493499b5eaf0acdeb5f9 (table default.Table_bf18dd7dd7984680bf41384772c15e61 [id=1af28dc40a8d408ba31c695e8563efc2]). Expected version 1 got 0
I20260927 23:28:06.941974 2388 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
I20260927 23:28:06.944013 2392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 4929fc9e396e48a189c94586d58aa55f
I20260927 23:28:06.945417 2392 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 163) took 1524 ms. Trace:
I20260927 23:28:06.945688 2392 rpcz_store.cc:276] 0927 23:28:05.421310 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:05.424508 (+ 3198us) service_pool.cc:227] Handling call
0927 23:28:05.449409 (+ 24901us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b3466 00007f8bb3bf4b34 00007f8bb3bf4ac2 00007f8bb3bf49f1 00007f8bb3bf496c 00007f8bb3bf3c22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:05.451282 (+ 1873us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:05.452139 (+ 857us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:28:05.452263 (+ 124us) catalog_manager.cc:2761] Locking tablets
0927 23:28:05.452757 (+ 494us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:28:05.819341 (+366584us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:28:05.819633 (+ 292us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:06.943916 (+1124283us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:06.945348 (+ 1432us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:05.796622 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:05.796960 (+ 338us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:05.796982 (+ 22us) write_op.cc:435] Acquired schema lock
0927 23:28:05.796994 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:05.797291 (+ 297us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:05.797330 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:05.797351 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:05.797598 (+ 247us) tablet.cc:699] Row locks acquired
0927 23:28:05.797611 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:28:05.797711 (+ 100us) write_op.cc:270] Start()
0927 23:28:05.797791 (+ 80us) write_op.cc:276] Timestamp: P: 1790551685797659 usec, L: 0
0927 23:28:05.797805 (+ 14us) op_driver.cc:331] REPLICATION: starting
0927 23:28:05.798257 (+ 452us) log.cc:842] Serialized 756 byte log entry
0927 23:28:05.817671 (+ 19414us) op_driver.cc:444] REPLICATION: finished
0927 23:28:05.818032 (+ 361us) write_op.cc:301] APPLY: starting
0927 23:28:05.818093 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:05.818199 (+ 106us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:05.818303 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:05.818516 (+ 213us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:05.818664 (+ 148us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:05.818706 (+ 42us) write_op.cc:312] APPLY: finished
0927 23:28:05.818883 (+ 177us) log.cc:842] Serialized 29 byte log entry
0927 23:28:05.819005 (+ 122us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:05.819069 (+ 64us) write_op.cc:454] Released schema lock
0927 23:28:05.819097 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:05.819271 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:05.819294 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":43678592,"child_traces":[["op",{"apply.queue_time_us":203,"apply.run_cpu_time_us":1426,"apply.run_wall_time_us":1426,"num_ops":2,"prepare.queue_time_us":341983,"prepare.run_cpu_time_us":2057,"prepare.run_wall_time_us":29690,"raft.queue_time_us":406,"raft.run_cpu_time_us":671,"raft.run_wall_time_us":16302,"replication_time_us":19789}]]}
I20260927 23:28:06.957504 2803 tablet_service.cc:1514] Processing DeleteTablet for tablet 4929fc9e396e48a189c94586d58aa55f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:05 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:06.945715 2948 tablet_service.cc:1514] Processing DeleteTablet for tablet 4929fc9e396e48a189c94586d58aa55f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:05 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:06.965072 3336 raft_consensus.cc:2229] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:06.970752 3559 ts_tablet_manager.cc:1905] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:06.973491 3167 tablet_bootstrap.cc:654] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:06.983690 2386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_678fca7e3fe641c39d8cdc436fc85d38" } new_table_name: "default.Table_0a327a97ce744e0783fc675f39f6d2fc" modify_external_catalogs: true
I20260927 23:28:06.989701 2803 tablet_service.cc:1467] Processing CreateTablet for tablet bbeeccc19a504e7984b293127766daf0 (DEFAULT_TABLE table=default.Table_27f9b741e6184aa0bb0f95975292a4ae [id=925b33a7e7404cd1bb45e7e9d9facfed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:06.995087 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 5c87d497da884544920b3be9c78daccf (DEFAULT_TABLE table=default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:06.994091 2948 tablet_service.cc:1467] Processing CreateTablet for tablet bbeeccc19a504e7984b293127766daf0 (DEFAULT_TABLE table=default.Table_27f9b741e6184aa0bb0f95975292a4ae [id=925b33a7e7404cd1bb45e7e9d9facfed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:06.998891 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 5c87d497da884544920b3be9c78daccf (DEFAULT_TABLE table=default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.000504 2807 tablet_service.cc:1467] Processing CreateTablet for tablet 9f4a9d69d75c4bfbb43e558b83b460d1 (DEFAULT_TABLE table=default.Table_e10895f6ce7143a4b328dfe1818e5c64 [id=13d7d633f0d04401855679055a1ac024]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.009904 2389 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:07.011034 2389 catalog_manager.cc:5495] T ef1b6b75dad24c609a10693728ba8e20 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:07.013844 2389 catalog_manager.cc:5495] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } }
I20260927 23:28:07.015213 2389 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
I20260927 23:28:07.021102 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbeeccc19a504e7984b293127766daf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.022874 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c87d497da884544920b3be9c78daccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.050144 3536 raft_consensus.cc:482] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:07.050850 3536 raft_consensus.cc:504] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.051740 2647 tablet_service.cc:1467] Processing CreateTablet for tablet bbeeccc19a504e7984b293127766daf0 (DEFAULT_TABLE table=default.Table_27f9b741e6184aa0bb0f95975292a4ae [id=925b33a7e7404cd1bb45e7e9d9facfed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.053051 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbeeccc19a504e7984b293127766daf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.053810 3536 leader_election.cc:288] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:07.055299 3070 ts_tablet_manager.cc:1428] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.136s user 0.008s sys 0.000s
I20260927 23:28:07.056102 3070 tablet_bootstrap.cc:492] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:07.056990 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 5c87d497da884544920b3be9c78daccf (DEFAULT_TABLE table=default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.057878 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c87d497da884544920b3be9c78daccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.061658 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet 4929fc9e396e48a189c94586d58aa55f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:05 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:07.079041 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 9f4a9d69d75c4bfbb43e558b83b460d1 (DEFAULT_TABLE table=default.Table_e10895f6ce7143a4b328dfe1818e5c64 [id=13d7d633f0d04401855679055a1ac024]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.082666 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f4a9d69d75c4bfbb43e558b83b460d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.087275 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3200d6bd5cb24ff5b1bbd59af9353dbe" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:07.097383 2386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1912a82ff46f4f9aaa24b7621cc3995d consensus state has no leader
W20260927 23:28:07.103845 2609 leader_election.cc:341] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:07.104779 3250 consensus_queue.cc:1035] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:07.106151 3250 consensus_queue.cc:1035] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:07.107402 3250 consensus_queue.cc:1035] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:07.022193 2389 catalog_manager.cc:5495] T 933d4c631a1345f9a117b87d0e669117 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } }
I20260927 23:28:07.115199 3167 tablet_bootstrap.cc:492] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:07.105224 2386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_b5538a7a7f4b4d49bad39084c724c90b" } modify_external_catalogs: true
I20260927 23:28:07.119735 2386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_b5538a7a7f4b4d49bad39084c724c90b" } modify_external_catalogs: true
I20260927 23:28:07.126297 2646 tablet_service.cc:1467] Processing CreateTablet for tablet a708a517b744496c83a3aa759b3f1081 (DEFAULT_TABLE table=default.Table_9e2be4d6dccd46449e29c796e22c2011 [id=0c99a649b8164ec98b568d9ce89e27f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.127478 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a708a517b744496c83a3aa759b3f1081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.119305 2806 tablet_service.cc:1467] Processing CreateTablet for tablet a708a517b744496c83a3aa759b3f1081 (DEFAULT_TABLE table=default.Table_9e2be4d6dccd46449e29c796e22c2011 [id=0c99a649b8164ec98b568d9ce89e27f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.135569 3167 ts_tablet_manager.cc:1397] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.402s user 0.011s sys 0.002s
I20260927 23:28:07.131995 2944 tablet_service.cc:1467] Processing CreateTablet for tablet 9f4a9d69d75c4bfbb43e558b83b460d1 (DEFAULT_TABLE table=default.Table_e10895f6ce7143a4b328dfe1818e5c64 [id=13d7d633f0d04401855679055a1ac024]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.139792 2944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f4a9d69d75c4bfbb43e558b83b460d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.143637 2946 tablet_service.cc:1467] Processing CreateTablet for tablet a708a517b744496c83a3aa759b3f1081 (DEFAULT_TABLE table=default.Table_9e2be4d6dccd46449e29c796e22c2011 [id=0c99a649b8164ec98b568d9ce89e27f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.144902 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a708a517b744496c83a3aa759b3f1081. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.144968 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0578b6c99c4209a8d3c76f2089ef46 (DEFAULT_TABLE table=default.Table_9f465f0d2ddc47b49fafeebc89e141bf [id=c218ad2d6eaf40e5be8dc8e16d1dc80b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.146454 2653 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0578b6c99c4209a8d3c76f2089ef46 (DEFAULT_TABLE table=default.Table_9f465f0d2ddc47b49fafeebc89e141bf [id=c218ad2d6eaf40e5be8dc8e16d1dc80b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.147732 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a0578b6c99c4209a8d3c76f2089ef46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.159283 2949 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0578b6c99c4209a8d3c76f2089ef46 (DEFAULT_TABLE table=default.Table_9f465f0d2ddc47b49fafeebc89e141bf [id=c218ad2d6eaf40e5be8dc8e16d1dc80b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.160604 2949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a0578b6c99c4209a8d3c76f2089ef46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.171864 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 50dd9944d3134ab9b23a6109fd7d6ee5 (DEFAULT_TABLE table=default.Table_4694990a288d48d5a3163f6c981c7bd5 [id=b21dcd8d42c14505beab3eee5bf39cfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.164264 3167 raft_consensus.cc:348] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.187457 3167 raft_consensus.cc:374] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:07.187752 3167 raft_consensus.cc:729] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:07.190410 3070 tablet_bootstrap.cc:700] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent opening tablet: real 0.130s user 0.004s sys 0.000s
I20260927 23:28:07.191191 3070 tablet_bootstrap.cc:654] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:07.192699 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 50dd9944d3134ab9b23a6109fd7d6ee5 (DEFAULT_TABLE table=default.Table_4694990a288d48d5a3163f6c981c7bd5 [id=b21dcd8d42c14505beab3eee5bf39cfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.193900 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50dd9944d3134ab9b23a6109fd7d6ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.227219 2654 tablet_service.cc:1467] Processing CreateTablet for tablet 371842420a6048fca3cbc02d18f74ffc (DEFAULT_TABLE table=default.Table_9c487cc2fe344578a7b995417de4aeac [id=80774107a7e649deb9818c4d206b7e0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.228408 2654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 371842420a6048fca3cbc02d18f74ffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.238811 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 56766ed12a7c4a668a7e08fc1ad39d2f (DEFAULT_TABLE table=default.Table_21343cdd532b49f4ad94e82d0187b8d1 [id=93f4f113ef3a440e8def28982a60af64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.240077 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56766ed12a7c4a668a7e08fc1ad39d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.245551 2386 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 44212aa8bed246e18e095691ac7e94f2
I20260927 23:28:07.257094 2387 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:07.257915 2387 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
W20260927 23:28:07.267563 2379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 38be781ead0e470e9928881594b4b98f (table default.Table_d609f64deb9647f79c581917277e1f93 [id=7148bae99bd941abb55f7141b044f9d1], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 38be781ead0e470e9928881594b4b98f consensus state has no leader
I20260927 23:28:07.268751 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:07.270757 2391 catalog_manager.cc:5495] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:07.273365 2391 catalog_manager.cc:5495] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } }
I20260927 23:28:07.279518 2649 tablet_service.cc:1467] Processing CreateTablet for tablet a1890852164d492db3540a97847888fd (DEFAULT_TABLE table=default.Table_49c3f2d8a1b14c02878df5f1934bbaba [id=1b4c77aab36b477593fa78f19be095af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.280902 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1890852164d492db3540a97847888fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.293008 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
I20260927 23:28:07.297458 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet cfbbe17f54d1418b95efc3943db0b6c1 (table default.Table_96276d84ee11493d89908e4ff16dc7fe [id=f21b53d8e90a48bc9cb7f6268c946cc1]). Expected version 1 got 0
I20260927 23:28:07.307799 2645 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb27269a590476f878ee52a2aefd141 (DEFAULT_TABLE table=default.Table_1cf287cdda1046558956f74f784c2536 [id=8beb1e4757014f69ba48b5ea31b27afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.309124 2645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb27269a590476f878ee52a2aefd141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.323385 2391 catalog_manager.cc:5495] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:07.340627 2644 tablet_service.cc:1467] Processing CreateTablet for tablet 1c87484b8b9742faa683f13d37308e58 (DEFAULT_TABLE table=default.Table_decc7c294d064d7daea036b101d04d55 [id=2cd91cbdb5b14e16b847253971c8cb25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.342180 2644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c87484b8b9742faa683f13d37308e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.368927 2608 leader_election.cc:302] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:07.369267 3250 raft_consensus.cc:482] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:07.369843 3516 raft_consensus.cc:2781] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:07.370219 3516 raft_consensus.cc:482] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:07.369946 3250 raft_consensus.cc:504] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.370565 3516 raft_consensus.cc:3037] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:07.372027 3250 leader_election.cc:288] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:07.377198 2945 tablet_service.cc:1467] Processing CreateTablet for tablet 371842420a6048fca3cbc02d18f74ffc (DEFAULT_TABLE table=default.Table_9c487cc2fe344578a7b995417de4aeac [id=80774107a7e649deb9818c4d206b7e0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.378497 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 371842420a6048fca3cbc02d18f74ffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.381963 2951 tablet_service.cc:1467] Processing CreateTablet for tablet 56766ed12a7c4a668a7e08fc1ad39d2f (DEFAULT_TABLE table=default.Table_21343cdd532b49f4ad94e82d0187b8d1 [id=93f4f113ef3a440e8def28982a60af64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.383203 2951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56766ed12a7c4a668a7e08fc1ad39d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.385124 2947 tablet_service.cc:1467] Processing CreateTablet for tablet a1890852164d492db3540a97847888fd (DEFAULT_TABLE table=default.Table_49c3f2d8a1b14c02878df5f1934bbaba [id=1b4c77aab36b477593fa78f19be095af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.386752 2943 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb27269a590476f878ee52a2aefd141 (DEFAULT_TABLE table=default.Table_1cf287cdda1046558956f74f784c2536 [id=8beb1e4757014f69ba48b5ea31b27afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.389909 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb27269a590476f878ee52a2aefd141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.380473 3516 raft_consensus.cc:504] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.393010 3516 leader_election.cc:288] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:07.394613 2906 consensus_peers.cc:487] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:07.188446 3167 consensus_queue.cc:260] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.399848 3167 ts_tablet_manager.cc:1428] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.264s user 0.004s sys 0.002s
I20260927 23:28:07.400862 3167 tablet_bootstrap.cc:492] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:07.213732 2799 tablet_service.cc:1467] Processing CreateTablet for tablet 371842420a6048fca3cbc02d18f74ffc (DEFAULT_TABLE table=default.Table_9c487cc2fe344578a7b995417de4aeac [id=80774107a7e649deb9818c4d206b7e0a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.232566 2798 tablet_service.cc:1467] Processing CreateTablet for tablet 56766ed12a7c4a668a7e08fc1ad39d2f (DEFAULT_TABLE table=default.Table_21343cdd532b49f4ad94e82d0187b8d1 [id=93f4f113ef3a440e8def28982a60af64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.269271 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbeeccc19a504e7984b293127766daf0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.279114 2800 tablet_service.cc:1467] Processing CreateTablet for tablet a1890852164d492db3540a97847888fd (DEFAULT_TABLE table=default.Table_49c3f2d8a1b14c02878df5f1934bbaba [id=1b4c77aab36b477593fa78f19be095af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.402458 2798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56766ed12a7c4a668a7e08fc1ad39d2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.403175 2800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1890852164d492db3540a97847888fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.403775 2799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 371842420a6048fca3cbc02d18f74ffc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.336030 2797 tablet_service.cc:1467] Processing CreateTablet for tablet 1c87484b8b9742faa683f13d37308e58 (DEFAULT_TABLE table=default.Table_decc7c294d064d7daea036b101d04d55 [id=2cd91cbdb5b14e16b847253971c8cb25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.409111 2797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c87484b8b9742faa683f13d37308e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.421010 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50dd9944d3134ab9b23a6109fd7d6ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.428627 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f4a9d69d75c4bfbb43e558b83b460d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.432073 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a708a517b744496c83a3aa759b3f1081. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:07.372516 2761 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.Ping -> {remote=127.2.57.60:41571, user_credentials={real_user=slave}} blocked reactor thread for 187893us
I20260927 23:28:07.435747 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a0578b6c99c4209a8d3c76f2089ef46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.183128 2821 raft_consensus.cc:3037] T ef1b6b75dad24c609a10693728ba8e20 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:07.322695 2802 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb27269a590476f878ee52a2aefd141 (DEFAULT_TABLE table=default.Table_1cf287cdda1046558956f74f784c2536 [id=8beb1e4757014f69ba48b5ea31b27afc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.438477 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb27269a590476f878ee52a2aefd141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.451375 2796 tablet_service.cc:1467] Processing CreateTablet for tablet 49883297d2b046b99f2ec1343c276ff9 (DEFAULT_TABLE table=default.Table_6568f80896124508811607305bf18ff9 [id=adbf504950b64a08996f102f4ac2d662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.452806 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49883297d2b046b99f2ec1343c276ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.456984 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c87d497da884544920b3be9c78daccf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.195966 2967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbbe17f54d1418b95efc3943db0b6c1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:07.463121 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 49883297d2b046b99f2ec1343c276ff9 (DEFAULT_TABLE table=default.Table_6568f80896124508811607305bf18ff9 [id=adbf504950b64a08996f102f4ac2d662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.464475 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1890852164d492db3540a97847888fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.464378 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49883297d2b046b99f2ec1343c276ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.466727 2942 tablet_service.cc:1467] Processing CreateTablet for tablet 1c87484b8b9742faa683f13d37308e58 (DEFAULT_TABLE table=default.Table_decc7c294d064d7daea036b101d04d55 [id=2cd91cbdb5b14e16b847253971c8cb25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.462404 3501 raft_consensus.cc:482] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:28:07.500694 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 131858us
W20260927 23:28:07.497334 3072 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.792s user 0.293s sys 0.439s
W20260927 23:28:07.502895 3072 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.798s user 0.293s sys 0.439s
I20260927 23:28:07.465327 3572 raft_consensus.cc:482] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:07.517449 3454 raft_consensus.cc:482] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:07.517946 3454 raft_consensus.cc:504] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.519943 3454 leader_election.cc:288] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:07.520864 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f71f948891b4d7fa95c3a1fbd92ce1e" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:07.523130 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04f670db808c4e289c655679da9d2d9d" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:07.530314 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e596517324300be30df8506a18904" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:07.553643 3070 tablet_bootstrap.cc:492] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:07.554167 3070 ts_tablet_manager.cc:1397] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.498s user 0.010s sys 0.000s
I20260927 23:28:07.572142 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04f670db808c4e289c655679da9d2d9d" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:07.575891 3570 raft_consensus.cc:482] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5ce671f028a34ea4b8f0520c2689c9a1)
I20260927 23:28:07.576843 3575 raft_consensus.cc:482] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:07.472477 2967 raft_consensus.cc:3037] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:07.585285 3336 raft_consensus.cc:2258] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:07.627008 2797 tablet_service.cc:1467] Processing CreateTablet for tablet 6d0d8a1f277242588f57e4df95df5d5a (DEFAULT_TABLE table=default.Table_0ec64b963d5643a291b169781879ee55 [id=7b1bf5063eef401eb9de9b6f7aa40e4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.629559 2797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d0d8a1f277242588f57e4df95df5d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.641458 3250 consensus_queue.cc:1035] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260927 23:28:07.639737 2649 tablet_service.cc:1467] Processing CreateTablet for tablet 6d0d8a1f277242588f57e4df95df5d5a (DEFAULT_TABLE table=default.Table_0ec64b963d5643a291b169781879ee55 [id=7b1bf5063eef401eb9de9b6f7aa40e4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.663862 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d0d8a1f277242588f57e4df95df5d5a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:07.643877 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 124210us
W20260927 23:28:07.668349 2609 leader_election.cc:341] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:07.669406 2609 leader_election.cc:341] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:07.641518 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 140053us
W20260927 23:28:07.674682 2608 consensus_peers.cc:487] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Couldn't send request to peer de216a4a8e4b4e1b857d8647a3b05f6d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7767a32c07454fe6a92154429a3b6899. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:07.677505 2608 leader_election.cc:302] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:07.695456 3536 raft_consensus.cc:2781] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:07.696004 3536 raft_consensus.cc:686] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:07.696894 3536 consensus_queue.cc:237] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.610730 3558 ts_tablet_manager.cc:1918] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:07.699304 3558 log.cc:1196] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/cc1c0da6ea5c4a09a71555f37becae33
I20260927 23:28:07.700196 3558 ts_tablet_manager.cc:1939] T cc1c0da6ea5c4a09a71555f37becae33 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
W20260927 23:28:07.706132 2609 consensus_peers.cc:487] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:07.719395 3540 raft_consensus.cc:482] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:07.719957 3540 raft_consensus.cc:504] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.722698 3540 leader_election.cc:288] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:07.724649 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_d0b2f5d4356249cd9d26d96db40990ac"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:07.726102 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d0b2f5d4356249cd9d26d96db40990ac in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:07.730640 2798 tablet_service.cc:1467] Processing CreateTablet for tablet 0a2f98dd905d40959dc676f94a4a595b (DEFAULT_TABLE table=default.Table_a902e331f690448a872f6c104857f2ca [id=1acae7217ece44919a8d247f5cf155d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.742626 3167 tablet_bootstrap.cc:654] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:07.498047 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 495076us
I20260927 23:28:07.683959 3070 raft_consensus.cc:348] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.755452 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 50dd9944d3134ab9b23a6109fd7d6ee5 (DEFAULT_TABLE table=default.Table_4694990a288d48d5a3163f6c981c7bd5 [id=b21dcd8d42c14505beab3eee5bf39cfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.513022 3501 raft_consensus.cc:504] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.752512 2968 raft_consensus.cc:2379] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:07.751534 2942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c87484b8b9742faa683f13d37308e58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:07.769093 2941 tablet_service.cc:1467] Processing CreateTablet for tablet 49883297d2b046b99f2ec1343c276ff9 (DEFAULT_TABLE table=default.Table_6568f80896124508811607305bf18ff9 [id=adbf504950b64a08996f102f4ac2d662]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.845211 3501 leader_election.cc:288] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:07.773862 2944 consensus_meta.cc:287] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent flushing consensus metadata: real 0.565s user 0.005s sys 0.005s
I20260927 23:28:07.739049 3570 raft_consensus.cc:504] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:07.847008 3570 leader_election.cc:288] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:07.739197 3575 raft_consensus.cc:504] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.848528 3575 leader_election.cc:288] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:07.788849 3336 ts_tablet_manager.cc:1905] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:07.798178 3072 tablet_bootstrap.cc:492] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:07.852757 3072 ts_tablet_manager.cc:1397] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 1.155s user 0.301s sys 0.440s
I20260927 23:28:07.854895 3072 raft_consensus.cc:348] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.855857 3072 raft_consensus.cc:374] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:07.856670 3072 raft_consensus.cc:729] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
W20260927 23:28:07.858085 2907 consensus_peers.cc:487] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:07.857385 3072 consensus_queue.cc:260] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.858843 3575 consensus_queue.cc:1035] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:07.860081 3072 ts_tablet_manager.cc:1428] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
W20260927 23:28:07.861018 2907 leader_election.cc:341] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 410294c28c49404e9c547b27111712eb
W20260927 23:28:07.812114 2949 consensus_meta.cc:287] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent flushing consensus metadata: real 0.601s user 0.009s sys 0.006s
I20260927 23:28:07.581997 3572 raft_consensus.cc:504] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.872115 3572 leader_election.cc:288] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:07.823395 2967 raft_consensus.cc:2454] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:07.874711 2940 tablet_service.cc:1467] Processing CreateTablet for tablet 6d0d8a1f277242588f57e4df95df5d5a (DEFAULT_TABLE table=default.Table_0ec64b963d5643a291b169781879ee55 [id=7b1bf5063eef401eb9de9b6f7aa40e4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.878544 3070 raft_consensus.cc:374] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:07.856680 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e596517324300be30df8506a18904" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:07.892616 2670 raft_consensus.cc:2454] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:07.857738 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f530db3b52944f6a283ee9e2bf66828" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:07.873144 2671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3200d6bd5cb24ff5b1bbd59af9353dbe" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:07.893726 2671 raft_consensus.cc:2454] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
W20260927 23:28:07.686636 2647 consensus_meta.cc:287] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent flushing consensus metadata: real 0.563s user 0.006s sys 0.001s
W20260927 23:28:07.689280 2648 consensus_meta.cc:287] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent flushing consensus metadata: real 0.585s user 0.010s sys 0.000s
I20260927 23:28:07.896487 3070 raft_consensus.cc:729] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:07.898125 2939 tablet_service.cc:1467] Processing CreateTablet for tablet 8cd0972f635d4979abf5fd30e9b02b1d (DEFAULT_TABLE table=default.Table_813e5d484f8f462b86e9136a57eebacd [id=b215370f7b594ab28c3e2a34de2a6f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.900208 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3200d6bd5cb24ff5b1bbd59af9353dbe" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:07.900785 2969 raft_consensus.cc:2454] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:07.905982 3070 consensus_queue.cc:260] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:07.749135 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac19099dd7bf4b669d504a654ee69482" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:07.849454 2826 raft_consensus.cc:3037] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:07.929219 3572 raft_consensus.cc:482] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader abe182b2afc64d908f692ef45eae1d51)
I20260927 23:28:07.929689 3572 raft_consensus.cc:504] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:07.931654 3572 leader_election.cc:288] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:07.738139 2799 tablet_service.cc:1467] Processing CreateTablet for tablet 8cd0972f635d4979abf5fd30e9b02b1d (DEFAULT_TABLE table=default.Table_813e5d484f8f462b86e9136a57eebacd [id=b215370f7b594ab28c3e2a34de2a6f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:07.974677 2386 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 189) took 1251 ms. Trace:
I20260927 23:28:07.974994 2386 rpcz_store.cc:276] 0927 23:28:06.723097 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:07.256459 (+533362us) service_pool.cc:227] Handling call
0927 23:28:07.724470 (+468011us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b3156 00007f8bb3bf0034 00007f8bb3beffc2 00007f8bb3befef1 00007f8bb3befe6c 00007f8bb3bef122 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:07.726456 (+ 1986us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:07.727776 (+ 1320us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:07.974087 (+246311us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:07.974353 (+ 266us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:07.974598 (+ 245us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:07.730437 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:07.730751 (+ 314us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:07.730774 (+ 23us) write_op.cc:435] Acquired schema lock
0927 23:28:07.730786 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:07.731027 (+ 241us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:07.731063 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:07.731084 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:07.731251 (+ 167us) tablet.cc:699] Row locks acquired
0927 23:28:07.731263 (+ 12us) write_op.cc:260] PREPARE: finished
0927 23:28:07.731363 (+ 100us) write_op.cc:270] Start()
0927 23:28:07.731452 (+ 89us) write_op.cc:276] Timestamp: P: 1790551687731317 usec, L: 0
0927 23:28:07.731467 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:07.731862 (+ 395us) log.cc:842] Serialized 433 byte log entry
0927 23:28:07.733706 (+ 1844us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b58000a01b0. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb405d834 00007f8bb4062771 00007f8bb40626f9 00007f8bb4062689 00007f8bb4062651 00007f8bb406181d 0000000000580784 00000000005805b9 00007f8bad18e6b6 00007f8bad191b11
0927 23:28:07.956569 (+222863us) op_driver.cc:444] REPLICATION: finished
0927 23:28:07.958767 (+ 2198us) write_op.cc:301] APPLY: starting
0927 23:28:07.958812 (+ 45us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:07.958900 (+ 88us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:07.958917 (+ 17us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:07.959256 (+ 339us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:07.959379 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:07.959417 (+ 38us) write_op.cc:312] APPLY: finished
0927 23:28:07.959572 (+ 155us) log.cc:842] Serialized 29 byte log entry
0927 23:28:07.959700 (+ 128us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:07.959762 (+ 62us) write_op.cc:454] Released schema lock
0927 23:28:07.959791 (+ 29us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":1073652480,"child_traces":[["op",{"apply.queue_time_us":2096,"num_ops":2,"prepare.queue_time_us":69,"prepare.run_cpu_time_us":1672,"prepare.run_wall_time_us":1850,"raft.queue_time_us":73,"raft.run_cpu_time_us":657,"raft.run_wall_time_us":1652,"replication_time_us":225042,"spinlock_wait_cycles":2540416,"threads_started":1}]]}
I20260927 23:28:07.981755 2672 raft_consensus.cc:2454] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:07.985615 3070 ts_tablet_manager.cc:1428] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.431s user 0.007s sys 0.000s
I20260927 23:28:07.986725 3070 tablet_bootstrap.cc:492] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:07.947311 2761 leader_election.cc:302] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:07.992321 2392 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:07.997505 3072 tablet_bootstrap.cc:492] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:07.999289 2392 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet cfbbe17f54d1418b95efc3943db0b6c1 (table default.Table_96276d84ee11493d89908e4ff16dc7fe [id=f21b53d8e90a48bc9cb7f6268c946cc1]). Expected version 1 got 0
I20260927 23:28:08.001680 2392 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
I20260927 23:28:08.003618 3072 tablet_bootstrap.cc:654] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:08.013183 2393 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_2ca15ccb4f1f4b388143d9810dbb4ec6 [id=29d80c3f3f21428b819d211d3e10f621] alter complete (version 2)
I20260927 23:28:08.044471 2386 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:08.037849 2798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a2f98dd905d40959dc676f94a4a595b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.059023 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 111) took 1395 ms. Trace:
I20260927 23:28:08.059352 3558 rpcz_store.cc:276] 0927 23:28:06.664049 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.664194 (+ 145us) service_pool.cc:227] Handling call
0927 23:28:06.684279 (+ 20085us) ts_tablet_manager.cc:1145] Deleting tablet cc1c0da6ea5c4a09a71555f37becae33
0927 23:28:06.684365 (+ 86us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:07.074414 (+390049us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:07.610389 (+535975us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.058959 (+448570us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":224159,"spinlock_wait_cycles":1341312,"tablet-delete.queue_time_us":19504,"thread_start_us":20426,"threads_started":1}
I20260927 23:28:08.060770 3558 tablet_replica.cc:330] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:08.061638 2823 raft_consensus.cc:3037] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:08.061915 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet cc1c0da6ea5c4a09a71555f37becae33 (table default.Table_29404653bfbb4e5d8192bcb604084df0 [id=baad95258e1a4358a58abc3fc1cc3737]) successfully deleted
I20260927 23:28:08.055929 2386 catalog_manager.cc:5495] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:08.065292 3558 raft_consensus.cc:2229] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:08.065836 3558 raft_consensus.cc:2258] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:08.066135 2386 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet cfbbe17f54d1418b95efc3943db0b6c1 (table default.Table_96276d84ee11493d89908e4ff16dc7fe [id=f21b53d8e90a48bc9cb7f6268c946cc1]). Expected version 1 got 0
I20260927 23:28:08.066453 3577 raft_consensus.cc:2726] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:08.066900 2386 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
I20260927 23:28:08.067083 2803 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42492 (request call id 114) took 1078 ms. Trace:
I20260927 23:28:08.068403 2391 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_bf18dd7dd7984680bf41384772c15e61 [id=1af28dc40a8d408ba31c695e8563efc2] alter complete (version 1)
I20260927 23:28:08.067368 2803 rpcz_store.cc:276] 0927 23:28:06.988828 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.988987 (+ 159us) service_pool.cc:227] Handling call
0927 23:28:07.268742 (+279755us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:07.268852 (+ 110us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:07.414131 (+145279us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:07.960882 (+546751us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:08.067034 (+106152us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":318782}
I20260927 23:28:08.070143 2390 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_3777b02a50e04654b10cc6bd6c73f51e [id=d3de7e8bfbed4409bf10ead542fe1c9d] alter complete (version 2)
I20260927 23:28:08.071054 2949 tablet_service.cc:1467] Processing CreateTablet for tablet f253a54ab3b44822aee266314fa45495 (DEFAULT_TABLE table=default.Table_3bae57755da641d491998ef6ab5dd4ec [id=7fadbf61f94e4214bea8977753469e8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.073047 2943 tablet_service.cc:1467] Processing CreateTablet for tablet 0a2f98dd905d40959dc676f94a4a595b (DEFAULT_TABLE table=default.Table_a902e331f690448a872f6c104857f2ca [id=1acae7217ece44919a8d247f5cf155d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.069707 2384 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_2b44b865509b414ba9c0c667dd764097 [id=3b845f646d7f4af99b7889442b4d7d0f] alter complete (version 1)
W20260927 23:28:08.082707 2385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 25) took 3392 ms (client timeout 1000 ms). Trace:
W20260927 23:28:08.084758 2385 rpcz_store.cc:269] 0927 23:28:04.690023 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:04.738615 (+ 48592us) service_pool.cc:227] Handling call
0927 23:28:08.082611 (+3343996us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:05.257853 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:05.258261 (+ 408us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:05.317725 (+ 59464us) write_op.cc:435] Acquired schema lock
0927 23:28:05.317738 (+ 13us) tablet.cc:660] Decoding operations
0927 23:28:05.318082 (+ 344us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:05.318117 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:05.318138 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:05.318385 (+ 247us) tablet.cc:699] Row locks acquired
0927 23:28:05.318399 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:05.318506 (+ 107us) write_op.cc:270] Start()
0927 23:28:05.318589 (+ 83us) write_op.cc:276] Timestamp: P: 1790551685318391 usec, L: 0
0927 23:28:05.318604 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:05.319094 (+ 490us) log.cc:842] Serialized 895 byte log entry
0927 23:28:05.363779 (+ 44685us) op_driver.cc:444] REPLICATION: finished
0927 23:28:05.363981 (+ 202us) write_op.cc:301] APPLY: starting
0927 23:28:05.364026 (+ 45us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:05.364121 (+ 95us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:05.364212 (+ 91us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:05.364461 (+ 249us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:05.364597 (+ 136us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:05.364637 (+ 40us) write_op.cc:312] APPLY: finished
0927 23:28:05.364816 (+ 179us) log.cc:842] Serialized 29 byte log entry
0927 23:28:05.364939 (+ 123us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:05.365012 (+ 73us) write_op.cc:454] Released schema lock
0927 23:28:05.365040 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:05.365190 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:05.365212 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":74,"apply.run_cpu_time_us":1398,"apply.run_wall_time_us":1397,"num_ops":2,"prepare.queue_time_us":81692,"prepare.run_cpu_time_us":2103,"prepare.run_wall_time_us":121510,"replication_time_us":45106}]]}
I20260927 23:28:08.091302 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e596517324300be30df8506a18904" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:08.091920 2969 raft_consensus.cc:2454] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:08.096645 2965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac19099dd7bf4b669d504a654ee69482" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:08.097247 2965 raft_consensus.cc:2454] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
W20260927 23:28:08.091643 2389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 26) took 2315 ms (client timeout 1000 ms). Trace:
W20260927 23:28:08.099224 2389 rpcz_store.cc:269] 0927 23:28:05.775967 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:05.777659 (+ 1692us) service_pool.cc:227] Handling call
0927 23:28:05.792536 (+ 14877us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:08.091583 (+2299047us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:07.150247 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:07.150616 (+ 369us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:07.150639 (+ 23us) write_op.cc:435] Acquired schema lock
0927 23:28:07.150653 (+ 14us) tablet.cc:660] Decoding operations
0927 23:28:07.151053 (+ 400us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:07.152544 (+ 1491us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:07.152567 (+ 23us) tablet.cc:683] Acquiring locks for 3 operations
0927 23:28:07.152817 (+ 250us) tablet.cc:699] Row locks acquired
0927 23:28:07.152831 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:07.152946 (+ 115us) write_op.cc:270] Start()
0927 23:28:07.153029 (+ 83us) write_op.cc:276] Timestamp: P: 1790551687152845 usec, L: 0
0927 23:28:07.153044 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:07.153529 (+ 485us) log.cc:842] Serialized 1266 byte log entry
0927 23:28:07.241852 (+ 88323us) op_driver.cc:444] REPLICATION: finished
0927 23:28:07.245471 (+ 3619us) write_op.cc:301] APPLY: starting
0927 23:28:07.245530 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:07.245650 (+ 120us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:07.245763 (+ 113us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:07.246091 (+ 328us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:07.246235 (+ 144us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0927 23:28:07.246275 (+ 40us) write_op.cc:312] APPLY: finished
0927 23:28:07.246484 (+ 209us) log.cc:842] Serialized 35 byte log entry
0927 23:28:07.247102 (+ 618us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:07.247176 (+ 74us) write_op.cc:454] Released schema lock
0927 23:28:07.247208 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:07.247377 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:07.247399 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":19,"spinlock_wait_cycles":27473792,"child_traces":[["op",{"apply.queue_time_us":3506,"apply.run_cpu_time_us":1647,"apply.run_wall_time_us":2103,"mutex_wait_us":7521,"num_ops":3,"prepare.queue_time_us":7626,"prepare.run_cpu_time_us":2133,"prepare.run_wall_time_us":6100,"raft.queue_time_us":953,"raft.run_cpu_time_us":12,"raft.run_wall_time_us":13,"replication_time_us":88743,"thread_start_us":390984,"threads_started":1}]]}
I20260927 23:28:08.102680 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_cccab8d895c740e0b88b2709527516e3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:08.103726 2967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f71f948891b4d7fa95c3a1fbd92ce1e" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:08.104290 2967 raft_consensus.cc:3037] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:08.106861 2393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 32) took 2421 ms (client timeout 1000 ms). Trace:
W20260927 23:28:08.104080 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cccab8d895c740e0b88b2709527516e3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:08.110193 3575 raft_consensus.cc:482] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:08.110642 3575 raft_consensus.cc:504] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.112495 3575 leader_election.cc:288] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:08.117321 2391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 27) took 1292 ms (client timeout 1000 ms). Trace:
W20260927 23:28:08.125304 2384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 31) took 3485 ms (client timeout 1000 ms). Trace:
W20260927 23:28:08.125638 2384 rpcz_store.cc:269] 0927 23:28:04.639472 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:04.650810 (+ 11338us) service_pool.cc:227] Handling call
0927 23:28:08.125012 (+3474202us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b2400003008. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bacfd2f11 00007f8bad156851 00007f8bad156702 00007f8bb5fedb09 00007f8badb7f480 00007f8badb8c3cf 00007f8bb60b26d4 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec
0927 23:28:08.125247 (+ 235us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:05.709642 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:05.710043 (+ 401us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:05.710067 (+ 24us) write_op.cc:435] Acquired schema lock
0927 23:28:05.710078 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:05.710342 (+ 264us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:05.710379 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:05.710404 (+ 25us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:05.710570 (+ 166us) tablet.cc:699] Row locks acquired
0927 23:28:05.710583 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:28:05.710709 (+ 126us) write_op.cc:270] Start()
0927 23:28:05.710793 (+ 84us) write_op.cc:276] Timestamp: P: 1790551685710661 usec, L: 0
0927 23:28:05.710808 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:05.711334 (+ 526us) log.cc:842] Serialized 356 byte log entry
0927 23:28:05.755108 (+ 43774us) op_driver.cc:444] REPLICATION: finished
0927 23:28:05.755325 (+ 217us) write_op.cc:301] APPLY: starting
0927 23:28:05.755380 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:05.755483 (+ 103us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:05.755585 (+ 102us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:05.755719 (+ 134us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:05.755804 (+ 85us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:05.755831 (+ 27us) write_op.cc:312] APPLY: finished
0927 23:28:05.755982 (+ 151us) log.cc:842] Serialized 23 byte log entry
0927 23:28:05.756112 (+ 130us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:05.756174 (+ 62us) write_op.cc:454] Released schema lock
0927 23:28:05.756202 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:05.756388 (+ 186us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:05.756407 (+ 19us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":30699,"reported_tablets":9,"spinlock_wait_cycles":6924928,"child_traces":[["op",{"apply.queue_time_us":77,"apply.run_cpu_time_us":1219,"apply.run_wall_time_us":1218,"num_ops":1,"prepare.queue_time_us":272150,"prepare.run_cpu_time_us":2105,"prepare.run_wall_time_us":32987,"raft.queue_time_us":1282,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":216,"replication_time_us":44214}]]}
I20260927 23:28:08.131795 2967 raft_consensus.cc:2454] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:08.144461 2945 tablet_service.cc:1467] Processing CreateTablet for tablet f58349edf0db40f98951e116fc2582b0 (DEFAULT_TABLE table=default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.146426 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.148021 2941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49883297d2b046b99f2ec1343c276ff9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.149333 2940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d0d8a1f277242588f57e4df95df5d5a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:08.107151 2393 rpcz_store.cc:269] 0927 23:28:05.685918 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:05.686070 (+ 152us) service_pool.cc:227] Handling call
0927 23:28:05.703294 (+ 17224us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:08.106803 (+2403509us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:06.866715 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:06.867086 (+ 371us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:06.867109 (+ 23us) write_op.cc:435] Acquired schema lock
0927 23:28:06.867120 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:06.867422 (+ 302us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:06.867467 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:06.867490 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:06.867636 (+ 146us) tablet.cc:699] Row locks acquired
0927 23:28:06.867650 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:06.867764 (+ 114us) write_op.cc:270] Start()
0927 23:28:06.867848 (+ 84us) write_op.cc:276] Timestamp: P: 1790551686867699 usec, L: 0
0927 23:28:06.867867 (+ 19us) op_driver.cc:331] REPLICATION: starting
0927 23:28:06.868485 (+ 618us) log.cc:842] Serialized 356 byte log entry
0927 23:28:06.877886 (+ 9401us) op_driver.cc:444] REPLICATION: finished
0927 23:28:06.881794 (+ 3908us) write_op.cc:301] APPLY: starting
0927 23:28:06.881858 (+ 64us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:06.881958 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:06.882041 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:06.882203 (+ 162us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:06.882332 (+ 129us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:06.882389 (+ 57us) write_op.cc:312] APPLY: finished
0927 23:28:06.882579 (+ 190us) log.cc:842] Serialized 23 byte log entry
0927 23:28:06.882726 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:06.882799 (+ 73us) write_op.cc:454] Released schema lock
0927 23:28:06.882832 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:06.883021 (+ 189us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:06.883045 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":14,"spinlock_wait_cycles":33393408,"child_traces":[["op",{"apply.queue_time_us":3738,"apply.run_cpu_time_us":1421,"apply.run_wall_time_us":1421,"num_ops":1,"prepare.queue_time_us":93,"prepare.run_cpu_time_us":2035,"prepare.run_wall_time_us":5454,"raft.queue_time_us":343,"raft.run_cpu_time_us":716,"raft.run_wall_time_us":717,"replication_time_us":9934}]]}
I20260927 23:28:08.150098 2393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_67d5684dd07845a3a70af79c67fed142" } new_table_name: "default.Table_238b25adc2074249b812f76518fd76df" modify_external_catalogs: true
I20260927 23:28:08.150509 2939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cd0972f635d4979abf5fd30e9b02b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.153288 2949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f253a54ab3b44822aee266314fa45495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.155459 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50dd9944d3134ab9b23a6109fd7d6ee5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:08.155849 2390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 17) took 4068 ms (client timeout 1000 ms). Trace:
I20260927 23:28:08.156747 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f58349edf0db40f98951e116fc2582b0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:08.170958 2392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 19) took 1267 ms (client timeout 1000 ms). Trace:
W20260927 23:28:08.171303 2392 rpcz_store.cc:269] 0927 23:28:06.904056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.953198 (+ 49142us) service_pool.cc:227] Handling call
0927 23:28:06.976584 (+ 23386us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:08.170891 (+1194307us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":28,"spinlock_wait_cycles":25915648}
I20260927 23:28:08.172674 2384 master_service.cc:946] ReplaceTablet: received request to replace tablet 2411d7c53a154fafad9edda9fa75bb36 from {username='slave'} at 127.0.0.1:51972
W20260927 23:28:08.117653 2391 rpcz_store.cc:269] 0927 23:28:06.825263 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.848105 (+ 22842us) service_pool.cc:227] Handling call
0927 23:28:08.117251 (+1269146us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:07.607230 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:07.607595 (+ 365us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:07.607620 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:28:07.607632 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:07.608080 (+ 448us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:07.608126 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:07.608147 (+ 21us) tablet.cc:683] Acquiring locks for 3 operations
0927 23:28:07.608442 (+ 295us) tablet.cc:699] Row locks acquired
0927 23:28:07.608456 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:07.608577 (+ 121us) write_op.cc:270] Start()
0927 23:28:07.608683 (+ 106us) write_op.cc:276] Timestamp: P: 1790551687608504 usec, L: 0
0927 23:28:07.608700 (+ 17us) op_driver.cc:331] REPLICATION: starting
0927 23:28:07.729082 (+120382us) log.cc:842] Serialized 1266 byte log entry
0927 23:28:07.729642 (+ 560us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068471 00007f8bb40df198 00007f8bb4540ade 00007f8bb45402b8 00007f8bb4544ba1 00007f8bb4544b59 00007f8bb4544ae9 00007f8bb4544ab1 00007f8bb4543dad 0000000000580784 00000000005805b9
0927 23:28:07.956338 (+226696us) op_driver.cc:444] REPLICATION: finished
0927 23:28:07.957174 (+ 836us) write_op.cc:301] APPLY: starting
0927 23:28:07.957227 (+ 53us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:07.957342 (+ 115us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:07.957364 (+ 22us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:07.957704 (+ 340us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:07.957835 (+ 131us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0927 23:28:07.957873 (+ 38us) write_op.cc:312] APPLY: finished
0927 23:28:07.958063 (+ 190us) log.cc:842] Serialized 35 byte log entry
0927 23:28:07.958204 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:07.958268 (+ 64us) write_op.cc:454] Released schema lock
0927 23:28:07.958298 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:07.958470 (+ 172us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:07.958495 (+ 25us) write_op.cc:454] Released schema lock
Related trace 'op':
0927 23:28:08.024365 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:08.024707 (+ 342us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:08.024728 (+ 21us) write_op.cc:435] Acquired schema lock
0927 23:28:08.024742 (+ 14us) tablet.cc:660] Decoding operations
0927 23:28:08.024973 (+ 231us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:08.025011 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:08.025030 (+ 19us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:08.025156 (+ 126us) tablet.cc:699] Row locks acquired
0927 23:28:08.025169 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:28:08.025269 (+ 100us) write_op.cc:270] Start()
0927 23:28:08.025347 (+ 78us) write_op.cc:276] Timestamp: P: 1790551688025197 usec, L: 0
0927 23:28:08.025363 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:08.025810 (+ 447us) log.cc:842] Serialized 356 byte log entry
0927 23:28:08.046897 (+ 21087us) op_driver.cc:444] REPLICATION: finished
0927 23:28:08.049603 (+ 2706us) write_op.cc:301] APPLY: starting
0927 23:28:08.049658 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:08.049744 (+ 86us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:08.049846 (+ 102us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:08.050016 (+ 170us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:08.050135 (+ 119us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:08.050179 (+ 44us) write_op.cc:312] APPLY: finished
0927 23:28:08.050365 (+ 186us) log.cc:842] Serialized 23 byte log entry
0927 23:28:08.050964 (+ 599us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.051027 (+ 63us) write_op.cc:454] Released schema lock
0927 23:28:08.051055 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:08.051305 (+ 250us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.051328 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":34,"reported_tablets":23,"child_traces":[["op",{"apply.queue_time_us":683,"apply.run_cpu_time_us":1505,"apply.run_wall_time_us":1505,"num_ops":3,"prepare.queue_time_us":111,"prepare.run_cpu_time_us":3852,"prepare.run_wall_time_us":122593,"raft.queue_time_us":2023,"raft.run_cpu_time_us":836,"raft.run_wall_time_us":148255,"replication_time_us":347543,"spinlock_wait_cycles":275707776}],["op",{"apply.queue_time_us":2543,"apply.run_cpu_time_us":1360,"apply.run_wall_time_us":1969,"num_ops":1,"prepare.queue_time_us":93,"prepare.run_cpu_time_us":1703,"prepare.run_wall_time_us":1704,"raft.queue_time_us":185,"raft.run_cpu_time_us":599,"raft.run_wall_time_us":1319,"replication_time_us":21461}]]}
I20260927 23:28:08.175221 2391 master_service.cc:946] ReplaceTablet: received request to replace tablet 3f71f948891b4d7fa95c3a1fbd92ce1e from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:08.179359 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_001a61d6538b4af19e206fc70f19f8f4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:08.182683 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_001a61d6538b4af19e206fc70f19f8f4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:08.190039 2388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 18) took 2590 ms (client timeout 1000 ms). Trace:
W20260927 23:28:08.190356 2388 rpcz_store.cc:269] 0927 23:28:05.600092 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:05.622971 (+ 22879us) service_pool.cc:227] Handling call
0927 23:28:08.189972 (+2567001us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":58350,"reported_tablets":26}
I20260927 23:28:08.191370 2388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_25f0e3650873407a8b3cbd8c8e0c86af" } modify_external_catalogs: true
I20260927 23:28:08.191847 2388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_25f0e3650873407a8b3cbd8c8e0c86af" } modify_external_catalogs: true
W20260927 23:28:08.193450 2387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 33) took 1460 ms (client timeout 1000 ms). Trace:
W20260927 23:28:08.193744 2387 rpcz_store.cc:269] 0927 23:28:06.732925 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.751243 (+ 18318us) service_pool.cc:227] Handling call
0927 23:28:06.979335 (+228092us) spinlock_profiling.cc:243] Waited 62.3 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:08.193377 (+1214042us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":66228,"reported_tablets":18,"spinlock_wait_cycles":143386880}
I20260927 23:28:08.194883 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_2342ac0b757f483bb8c19981aab80740"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:08.196277 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2342ac0b757f483bb8c19981aab80740 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:08.156158 2390 rpcz_store.cc:269] 0927 23:28:04.087838 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:04.087944 (+ 106us) service_pool.cc:227] Handling call
0927 23:28:08.155785 (+4067841us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:05.661091 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:05.661450 (+ 359us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:05.661473 (+ 23us) write_op.cc:435] Acquired schema lock
0927 23:28:05.661485 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:05.661761 (+ 276us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:05.661803 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:05.661824 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:05.661966 (+ 142us) tablet.cc:699] Row locks acquired
0927 23:28:05.661978 (+ 12us) write_op.cc:260] PREPARE: finished
0927 23:28:05.662080 (+ 102us) write_op.cc:270] Start()
0927 23:28:05.662164 (+ 84us) write_op.cc:276] Timestamp: P: 1790551685662041 usec, L: 0
0927 23:28:05.662180 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:05.662674 (+ 494us) log.cc:842] Serialized 356 byte log entry
0927 23:28:05.742592 (+ 79918us) op_driver.cc:444] REPLICATION: finished
0927 23:28:05.743771 (+ 1179us) write_op.cc:301] APPLY: starting
0927 23:28:05.743828 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:05.743919 (+ 91us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:05.744006 (+ 87us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:05.744168 (+ 162us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:05.744282 (+ 114us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:05.744340 (+ 58us) write_op.cc:312] APPLY: finished
0927 23:28:05.744537 (+ 197us) log.cc:842] Serialized 23 byte log entry
0927 23:28:05.744673 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:05.744727 (+ 54us) write_op.cc:454] Released schema lock
0927 23:28:05.744761 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:05.744920 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:05.744942 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":36631,"reported_tablets":26,"child_traces":[["op",{"apply.queue_time_us":1026,"apply.run_cpu_time_us":1323,"apply.run_wall_time_us":1322,"num_ops":1,"prepare.queue_time_us":226564,"prepare.run_cpu_time_us":2016,"prepare.run_wall_time_us":48438,"raft.queue_time_us":322,"raft.run_cpu_time_us":684,"raft.run_wall_time_us":1049,"replication_time_us":80322}]]}
I20260927 23:28:08.217407 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a2f98dd905d40959dc676f94a4a595b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.218572 2387 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 196) took 1033 ms. Trace:
I20260927 23:28:08.218909 2387 rpcz_store.cc:276] 0927 23:28:07.185309 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.194485 (+1009176us) service_pool.cc:227] Handling call
0927 23:28:08.196705 (+ 2220us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:08.197988 (+ 1283us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:08.218001 (+ 20013us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:08.218271 (+ 270us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:08.218498 (+ 227us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:08.199107 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:08.199416 (+ 309us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:08.199448 (+ 32us) write_op.cc:435] Acquired schema lock
0927 23:28:08.199461 (+ 13us) tablet.cc:660] Decoding operations
0927 23:28:08.199709 (+ 248us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:08.199744 (+ 35us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:08.199763 (+ 19us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:08.199951 (+ 188us) tablet.cc:699] Row locks acquired
0927 23:28:08.199964 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:28:08.200059 (+ 95us) write_op.cc:270] Start()
0927 23:28:08.200132 (+ 73us) write_op.cc:276] Timestamp: P: 1790551688199969 usec, L: 0
0927 23:28:08.200147 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:08.200644 (+ 497us) log.cc:842] Serialized 433 byte log entry
0927 23:28:08.209705 (+ 9061us) op_driver.cc:444] REPLICATION: finished
0927 23:28:08.211312 (+ 1607us) write_op.cc:301] APPLY: starting
0927 23:28:08.211358 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:08.211442 (+ 84us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:08.211528 (+ 86us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:08.212777 (+ 1249us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:08.212900 (+ 123us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:08.212953 (+ 53us) write_op.cc:312] APPLY: finished
0927 23:28:08.213108 (+ 155us) log.cc:842] Serialized 29 byte log entry
0927 23:28:08.213227 (+ 119us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.213285 (+ 58us) write_op.cc:454] Released schema lock
0927 23:28:08.213316 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:08.213461 (+ 145us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.213481 (+ 20us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1508,"apply.run_cpu_time_us":2078,"apply.run_wall_time_us":2294,"num_ops":2,"prepare.queue_time_us":89,"prepare.run_cpu_time_us":1760,"prepare.run_wall_time_us":1774,"raft.queue_time_us":1917,"raft.run_cpu_time_us":9,"raft.run_wall_time_us":12,"replication_time_us":9501}]]}
I20260927 23:28:08.225589 2390 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_777042bde16f4fc4a33505d169021646" } new_table_name: "default.Table_a5f6638fb0a940fcb37e20213e3d081f" modify_external_catalogs: true
I20260927 23:28:08.237372 2385 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:08.245937 2385 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 1 got 0
I20260927 23:28:08.248374 2392 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 192) took 1295 ms. Trace:
I20260927 23:28:08.249132 2384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 2411d7c53a154fafad9edda9fa75bb36
I20260927 23:28:08.250684 2384 catalog_manager.cc:6565] ReplaceTablet: tablet 2411d7c53a154fafad9edda9fa75bb36 deleted and replaced by tablet 73cfb41befe44e5189029abafe7e326d
I20260927 23:28:08.251865 2389 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 190) took 1436 ms. Trace:
I20260927 23:28:08.248833 2392 rpcz_store.cc:276] 0927 23:28:06.953003 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.172029 (+1219026us) service_pool.cc:227] Handling call
0927 23:28:08.183039 (+ 11010us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:08.223046 (+ 40007us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:08.247800 (+ 24754us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:08.248065 (+ 265us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:08.248320 (+ 255us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:08.229932 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:08.230230 (+ 298us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:08.230250 (+ 20us) write_op.cc:435] Acquired schema lock
0927 23:28:08.230263 (+ 13us) tablet.cc:660] Decoding operations
0927 23:28:08.230519 (+ 256us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:08.230549 (+ 30us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:08.230571 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:08.230737 (+ 166us) tablet.cc:699] Row locks acquired
0927 23:28:08.230749 (+ 12us) write_op.cc:260] PREPARE: finished
0927 23:28:08.230839 (+ 90us) write_op.cc:270] Start()
0927 23:28:08.230907 (+ 68us) write_op.cc:276] Timestamp: P: 1790551688230811 usec, L: 0
0927 23:28:08.230922 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:08.231305 (+ 383us) log.cc:842] Serialized 433 byte log entry
0927 23:28:08.244446 (+ 13141us) op_driver.cc:444] REPLICATION: finished
0927 23:28:08.246249 (+ 1803us) write_op.cc:301] APPLY: starting
0927 23:28:08.246295 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:08.246376 (+ 81us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:08.246453 (+ 77us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:08.246967 (+ 514us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:08.247089 (+ 122us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:08.247126 (+ 37us) write_op.cc:312] APPLY: finished
0927 23:28:08.247268 (+ 142us) log.cc:842] Serialized 29 byte log entry
0927 23:28:08.247385 (+ 117us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.247446 (+ 61us) write_op.cc:454] Released schema lock
0927 23:28:08.247474 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:08.247615 (+ 141us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.247636 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1099,"apply.run_cpu_time_us":1400,"apply.run_wall_time_us":1519,"mutex_wait_us":2199,"num_ops":2,"prepare.queue_time_us":5900,"raft.queue_time_us":2735,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":7,"replication_time_us":13458,"threads_started":1}]]}
I20260927 23:28:08.252324 2389 rpcz_store.cc:276] 0927 23:28:06.815678 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.102273 (+1286595us) service_pool.cc:227] Handling call
0927 23:28:08.107766 (+ 5493us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:08.211418 (+103652us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:08.251105 (+ 39687us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:08.251497 (+ 392us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:08.251800 (+ 303us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:08.226367 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:08.226709 (+ 342us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:08.226731 (+ 22us) write_op.cc:435] Acquired schema lock
0927 23:28:08.226743 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:08.226988 (+ 245us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:08.227025 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:08.227045 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:08.227218 (+ 173us) tablet.cc:699] Row locks acquired
0927 23:28:08.227230 (+ 12us) write_op.cc:260] PREPARE: finished
0927 23:28:08.227330 (+ 100us) write_op.cc:270] Start()
0927 23:28:08.227399 (+ 69us) write_op.cc:276] Timestamp: P: 1790551688227302 usec, L: 0
0927 23:28:08.227414 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:08.227867 (+ 453us) log.cc:842] Serialized 433 byte log entry
0927 23:28:08.241646 (+ 13779us) op_driver.cc:444] REPLICATION: finished
0927 23:28:08.242039 (+ 393us) write_op.cc:301] APPLY: starting
0927 23:28:08.242090 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:08.242190 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:08.242269 (+ 79us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:08.242649 (+ 380us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:08.242791 (+ 142us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:08.242829 (+ 38us) write_op.cc:312] APPLY: finished
0927 23:28:08.243007 (+ 178us) log.cc:842] Serialized 29 byte log entry
0927 23:28:08.243142 (+ 135us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.243197 (+ 55us) write_op.cc:454] Released schema lock
0927 23:28:08.243231 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:08.243406 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.243428 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":249,"apply.run_cpu_time_us":1531,"apply.run_wall_time_us":1531,"num_ops":2,"prepare.queue_time_us":13881,"prepare.run_cpu_time_us":1736,"prepare.run_wall_time_us":1737,"raft.queue_time_us":4077,"replication_time_us":14158}]]}
I20260927 23:28:08.257221 2385 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet cfbbe17f54d1418b95efc3943db0b6c1 (table default.Table_96276d84ee11493d89908e4ff16dc7fe [id=f21b53d8e90a48bc9cb7f6268c946cc1]). Expected version 1 got 0
I20260927 23:28:08.258601 2385 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
I20260927 23:28:08.251986 2384 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:51972 (request call id 193) took 1282 ms. Trace:
I20260927 23:28:08.267295 3336 ts_tablet_manager.cc:1918] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:08.267699 3336 log.cc:1196] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/3acdfe7faa1a4d5d8e340f09f1cae93e
I20260927 23:28:08.268800 2393 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:51972 (request call id 191) took 1446 ms. Trace:
I20260927 23:28:08.259570 2384 rpcz_store.cc:276] 0927 23:28:06.969978 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.172567 (+1202589us) service_pool.cc:227] Handling call
0927 23:28:08.213149 (+ 40582us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b5b81 00007f8bb3c10d37 00007f8bb3c10cc2 00007f8bb3c10bf1 00007f8bb3c10b6c 00007f8bb3c0fe22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:08.251920 (+ 38771us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:08.228186 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:08.228524 (+ 338us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:08.228544 (+ 20us) write_op.cc:435] Acquired schema lock
0927 23:28:08.228555 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:08.228808 (+ 253us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:08.228840 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:08.228859 (+ 19us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:08.229025 (+ 166us) tablet.cc:699] Row locks acquired
0927 23:28:08.229036 (+ 11us) write_op.cc:260] PREPARE: finished
0927 23:28:08.229124 (+ 88us) write_op.cc:270] Start()
0927 23:28:08.229192 (+ 68us) write_op.cc:276] Timestamp: P: 1790551688229096 usec, L: 0
0927 23:28:08.229206 (+ 14us) op_driver.cc:331] REPLICATION: starting
0927 23:28:08.229636 (+ 430us) log.cc:842] Serialized 619 byte log entry
0927 23:28:08.241847 (+ 12211us) op_driver.cc:444] REPLICATION: finished
0927 23:28:08.243685 (+ 1838us) write_op.cc:301] APPLY: starting
0927 23:28:08.243733 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:08.243832 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:08.243923 (+ 91us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:08.244246 (+ 323us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:08.244826 (+ 580us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:08.244865 (+ 39us) write_op.cc:312] APPLY: finished
0927 23:28:08.245017 (+ 152us) log.cc:842] Serialized 29 byte log entry
0927 23:28:08.245140 (+ 123us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.245194 (+ 54us) write_op.cc:454] Released schema lock
0927 23:28:08.245226 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:08.245370 (+ 144us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.245393 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":23598464,"child_traces":[["op",{"apply.queue_time_us":1729,"apply.run_cpu_time_us":1512,"apply.run_wall_time_us":1870,"mutex_wait_us":10345,"num_ops":2,"prepare.queue_time_us":12384,"prepare.run_cpu_time_us":1662,"prepare.run_wall_time_us":1660,"raft.queue_time_us":4666,"raft.run_cpu_time_us":11,"raft.run_wall_time_us":12,"replication_time_us":12579}]]}
I20260927 23:28:08.269220 2393 rpcz_store.cc:276] 0927 23:28:06.821976 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.149793 (+1327817us) service_pool.cc:227] Handling call
0927 23:28:08.150665 (+ 872us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:08.151491 (+ 826us) catalog_manager.cc:3616] Apply alter schema
0927 23:28:08.165602 (+ 14111us) catalog_manager.cc:3639] Acquired catalog manager lock
0927 23:28:08.165832 (+ 230us) catalog_manager.cc:3786] Updating metadata on disk
0927 23:28:08.216943 (+ 51111us) catalog_manager.cc:3817] Committing alterations to in-memory state
0927 23:28:08.268743 (+ 51800us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:08.167044 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:08.167369 (+ 325us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:08.167393 (+ 24us) write_op.cc:435] Acquired schema lock
0927 23:28:08.167405 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:08.167658 (+ 253us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:08.167698 (+ 40us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:08.167719 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:08.167846 (+ 127us) tablet.cc:699] Row locks acquired
0927 23:28:08.167860 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:08.167961 (+ 101us) write_op.cc:270] Start()
0927 23:28:08.168036 (+ 75us) write_op.cc:276] Timestamp: P: 1790551688167873 usec, L: 0
0927 23:28:08.168051 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:08.168528 (+ 477us) log.cc:842] Serialized 405 byte log entry
0927 23:28:08.209523 (+ 40995us) op_driver.cc:444] REPLICATION: finished
0927 23:28:08.209953 (+ 430us) write_op.cc:301] APPLY: starting
0927 23:28:08.210004 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:08.210085 (+ 81us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:08.210179 (+ 94us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:08.210309 (+ 130us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:08.210447 (+ 138us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:08.210486 (+ 39us) write_op.cc:312] APPLY: finished
0927 23:28:08.210639 (+ 153us) log.cc:842] Serialized 23 byte log entry
0927 23:28:08.210772 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.210826 (+ 54us) write_op.cc:454] Released schema lock
0927 23:28:08.210854 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:08.211033 (+ 179us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.211056 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5438,"child_traces":[["op",{"apply.queue_time_us":297,"apply.run_cpu_time_us":1252,"apply.run_wall_time_us":1251,"num_ops":1,"prepare.queue_time_us":98,"prepare.run_cpu_time_us":1723,"prepare.run_wall_time_us":1725,"raft.queue_time_us":33191,"raft.run_cpu_time_us":727,"raft.run_wall_time_us":727,"replication_time_us":41403}]]}
I20260927 23:28:08.281853 3072 tablet_bootstrap.cc:492] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
W20260927 23:28:08.284443 2377 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1912a82ff46f4f9aaa24b7621cc3995d consensus state has no leader
I20260927 23:28:08.286715 3072 ts_tablet_manager.cc:1397] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.289s user 0.010s sys 0.000s
I20260927 23:28:08.287619 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet 45dd55b5a9d94ff0b84a799174ab5359 with delete_type TABLET_DATA_DELETED (replaced by tablet 2d0d8a82c5234880ab8396b1777e36c8) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.292331 3072 raft_consensus.cc:348] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.308049 3072 raft_consensus.cc:374] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:08.308446 3072 raft_consensus.cc:729] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:08.309093 3072 consensus_queue.cc:260] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.305260 3336 ts_tablet_manager.cc:1939] T 3acdfe7faa1a4d5d8e340f09f1cae93e P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:08.314832 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 92) took 4345 ms. Trace:
I20260927 23:28:08.317471 2804 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42492 (request call id 115) took 1323 ms. Trace:
I20260927 23:28:08.318068 2804 rpcz_store.cc:276] 0927 23:28:06.994346 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.994509 (+ 163us) service_pool.cc:227] Handling call
0927 23:28:07.269485 (+274976us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:07.269576 (+ 91us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:07.942183 (+672607us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.310088 (+367905us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:08.317412 (+ 7324us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:08.323127 2387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_0b205c7018ea4694b24bcfaeb93e859a" } modify_external_catalogs: true
I20260927 23:28:08.323613 2387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_0b205c7018ea4694b24bcfaeb93e859a" } modify_external_catalogs: true
I20260927 23:28:08.327517 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_477638f0398348fdbb82d985e67e7c14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:08.329130 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_477638f0398348fdbb82d985e67e7c14 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:08.319192 2800 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42492 (request call id 122) took 1040 ms. Trace:
I20260927 23:28:08.340847 2802 tablet_service.cc:1467] Processing CreateTablet for tablet f253a54ab3b44822aee266314fa45495 (DEFAULT_TABLE table=default.Table_3bae57755da641d491998ef6ab5dd4ec [id=7fadbf61f94e4214bea8977753469e8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.329653 2799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cd0972f635d4979abf5fd30e9b02b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.346179 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f530db3b52944f6a283ee9e2bf66828" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:08.347646 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f253a54ab3b44822aee266314fa45495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.349931 2807 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42492 (request call id 116) took 1350 ms. Trace:
I20260927 23:28:08.350630 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e596517324300be30df8506a18904" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:08.350837 2806 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42492 (request call id 117) took 1233 ms. Trace:
I20260927 23:28:08.351107 2806 rpcz_store.cc:276] 0927 23:28:07.117224 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:07.118595 (+ 1371us) service_pool.cc:227] Handling call
0927 23:28:07.269997 (+151402us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:07.270086 (+ 89us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:07.944516 (+674430us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.315418 (+370902us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:08.350792 (+ 35374us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:08.316723 3072 ts_tablet_manager.cc:1428] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20260927 23:28:08.352845 3072 tablet_bootstrap.cc:492] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:08.334286 2942 tablet_service.cc:1514] Processing DeleteTablet for tablet a80bff324902498fa5c04fe8cbc05f9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:02 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.335062 2944 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cce47b050b445dfba7be5ea3b6a8636 with delete_type TABLET_DATA_DELETED (replaced by tablet 1da38499af164d7fb5c1a8d65167ad24) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.354317 2803 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cce47b050b445dfba7be5ea3b6a8636 with delete_type TABLET_DATA_DELETED (replaced by tablet 1da38499af164d7fb5c1a8d65167ad24) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.335796 2951 tablet_service.cc:1514] Processing DeleteTablet for tablet 66f2842218fc42e5b1b83314e5f1a158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.355505 2797 tablet_service.cc:1514] Processing DeleteTablet for tablet 5efbd0211cf3408cb5ea9640d8da6694 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:02 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.355679 2803 tablet_service.cc:1467] Processing CreateTablet for tablet f58349edf0db40f98951e116fc2582b0 (DEFAULT_TABLE table=default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.356146 2795 tablet_service.cc:1514] Processing DeleteTablet for tablet 7096e434ac9f4290ad58ba4f7e7c2513 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.356703 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f58349edf0db40f98951e116fc2582b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.365316 2805 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42492 (request call id 118) took 1221 ms. Trace:
I20260927 23:28:08.352222 3336 rpcz_store.cc:276] 0927 23:28:03.969126 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:03.969213 (+ 87us) service_pool.cc:227] Handling call
0927 23:28:06.937753 (+2968540us) ts_tablet_manager.cc:1145] Deleting tablet 3acdfe7faa1a4d5d8e340f09f1cae93e
0927 23:28:06.937807 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:07.498826 (+561019us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b58000608b0. stack: 00007f5a8686c033 00007f5a8686bf30 00007f5a863ed534 00000000004db2b4 00000000004db249 00007f5a8d778471 00007f5a8d76ef63 00007f5a8d7d2aba 00007f5a8d7fb375 00007f5a8dc1126c 00007f5a8e81ecfa 00007f5a8e84ee10 00007f5a8e835e0d 00007f5a8e835db9 00007f5a8e835d49 00007f5a8e835d11
0927 23:28:07.864884 (+366058us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.267069 (+402185us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.314765 (+ 47696us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":507666688,"tablet-delete.queue_time_us":2949579}
I20260927 23:28:08.353842 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet 5efbd0211cf3408cb5ea9640d8da6694 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:02 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.365631 2805 rpcz_store.cc:276] 0927 23:28:07.144050 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:07.144221 (+ 171us) service_pool.cc:227] Handling call
0927 23:28:07.270243 (+126022us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:07.270329 (+ 86us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:07.941866 (+671537us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.360499 (+418633us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:08.365241 (+ 4742us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:08.350230 2807 rpcz_store.cc:276] 0927 23:28:06.999585 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.999736 (+ 151us) service_pool.cc:227] Handling call
0927 23:28:07.269739 (+270003us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:07.269844 (+ 105us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:07.921528 (+651684us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.326575 (+405047us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:08.349879 (+ 23304us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:08.379961 2385 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 84d37dba66b84d05abf675d3eaf39a30 (table default.Table_238b25adc2074249b812f76518fd76df [id=6671aa9b745f4863982b1d2dc75e5b63]). Expected version 1 got 0
I20260927 23:28:08.382436 3336 tablet_replica.cc:330] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:08.383265 3336 raft_consensus.cc:2229] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:08.383960 3577 raft_consensus.cc:482] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:08.384410 3577 raft_consensus.cc:504] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.387495 3577 leader_election.cc:288] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:08.371170 2796 tablet_service.cc:1514] Processing DeleteTablet for tablet 66f2842218fc42e5b1b83314e5f1a158 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.388327 2794 tablet_service.cc:1467] Processing CreateTablet for tablet db8996200d9c4c5f8572d1db2dbfe4e3 (DEFAULT_TABLE table=default.Table_511501de200744a59414f1dd8bf6b2fa [id=1d8e474b764741da8c4692ab994e26b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.390998 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db8996200d9c4c5f8572d1db2dbfe4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.395385 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]). Expected version 1 got 0
I20260927 23:28:08.395931 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
I20260927 23:28:08.396443 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 84d37dba66b84d05abf675d3eaf39a30 (table default.Table_238b25adc2074249b812f76518fd76df [id=6671aa9b745f4863982b1d2dc75e5b63]). Expected version 1 got 0
I20260927 23:28:08.397588 3336 raft_consensus.cc:2258] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:08.400058 2906 leader_election.cc:341] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Not found: Tablet not found: 410294c28c49404e9c547b27111712eb
I20260927 23:28:08.400491 2906 leader_election.cc:302] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:08.402606 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "702ebdeb7c5a4b7d900dbb00236cb8ae" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:08.403656 3072 tablet_bootstrap.cc:654] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:08.405747 2906 raft_consensus.cc:2670] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260927 23:28:08.407181 2906 leader_election.cc:341] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:08.409015 2906 leader_election.cc:341] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:08.410565 3575 consensus_queue.cc:1035] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:08.411015 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3200d6bd5cb24ff5b1bbd59af9353dbe" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:08.412760 2796 tablet_service.cc:1514] Processing DeleteTablet for tablet 7767a32c07454fe6a92154429a3b6899 with delete_type TABLET_DATA_DELETED (replaced by tablet 49883297d2b046b99f2ec1343c276ff9) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.412894 2941 tablet_service.cc:1514] Processing DeleteTablet for tablet 7767a32c07454fe6a92154429a3b6899 with delete_type TABLET_DATA_DELETED (replaced by tablet 49883297d2b046b99f2ec1343c276ff9) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.399978 2939 tablet_service.cc:1514] Processing DeleteTablet for tablet 828f870365244777b59e8a0756d7d4c9 with delete_type TABLET_DATA_DELETED (replaced by tablet 5bf856a05dff4fb4943fe8e1d9b7f385) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.414881 2796 tablet_service.cc:1467] Processing CreateTablet for tablet 119a1338b04740d0bbf291c8fbe34992 (DEFAULT_TABLE table=default.Table_91684f2e139444c0a8175fb9d1998733 [id=9677fe3601944b80ad51f032eae8069e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.415853 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 119a1338b04740d0bbf291c8fbe34992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.417412 3336 ts_tablet_manager.cc:1905] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:08.419369 2942 tablet_service.cc:1467] Processing CreateTablet for tablet ee5d073c8df84f0884dc0f0d88eeed9f (DEFAULT_TABLE table=default.Table_92117e5358444ef7bb1d38724fd8f933 [id=7163a6fa245d4c3b85f113b2786872dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.420950 2945 tablet_service.cc:1467] Processing CreateTablet for tablet 119a1338b04740d0bbf291c8fbe34992 (DEFAULT_TABLE table=default.Table_91684f2e139444c0a8175fb9d1998733 [id=9677fe3601944b80ad51f032eae8069e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.405668 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet 7096e434ac9f4290ad58ba4f7e7c2513 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:27:58 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.421720 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 3acdfe7faa1a4d5d8e340f09f1cae93e (table default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2]) successfully deleted
I20260927 23:28:08.422655 2939 tablet_service.cc:1514] Processing DeleteTablet for tablet 2411d7c53a154fafad9edda9fa75bb36 with delete_type TABLET_DATA_DELETED (replaced by tablet 73cfb41befe44e5189029abafe7e326d) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.423723 2941 tablet_service.cc:1467] Processing CreateTablet for tablet db8996200d9c4c5f8572d1db2dbfe4e3 (DEFAULT_TABLE table=default.Table_511501de200744a59414f1dd8bf6b2fa [id=1d8e474b764741da8c4692ab994e26b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.425241 2801 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:42492 (request call id 119) took 1254 ms. Trace:
I20260927 23:28:08.431535 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4277045ec344ad7a48db9b14bea5e7b" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:08.433000 2942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee5d073c8df84f0884dc0f0d88eeed9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.436228 2939 tablet_service.cc:1467] Processing CreateTablet for tablet 31676cfbbad04e728adb4b8aa8d905a6 (DEFAULT_TABLE table=default.Table_3f55d539f6264cef8baea2cc68a8fe37 [id=df3186e81f1a4ea0977995aeef912b77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.437553 2949 tablet_service.cc:1467] Processing CreateTablet for tablet a8df6d0652634def9c8946c346dea702 (DEFAULT_TABLE table=default.Table_4dd942a54bfc41a882f7e66a38e4d1fb [id=996f8a130c7f48b4b23b74dad8f5fee4]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:08.429667 2906 leader_election.cc:341] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:08.441821 2906 leader_election.cc:341] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:08.442315 2906 leader_election.cc:341] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Not found: Tablet not found: 74fc2f03baae4f70b38b8079deb8a854
I20260927 23:28:08.443087 2384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_732525debefe48c49e2657fdd56b038b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:08.451395 2388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 5bf856a05dff4fb4943fe8e1d9b7f385
I20260927 23:28:08.400695 2950 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44952 (request call id 125) took 1192 ms. Trace:
I20260927 23:28:08.453034 2950 rpcz_store.cc:276] 0927 23:28:07.208102 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:07.740849 (+532747us) service_pool.cc:227] Handling call
0927 23:28:08.004348 (+263499us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:08.004721 (+ 373us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:08.225167 (+220446us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.397951 (+172784us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:08.400648 (+ 2697us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:08.453909 2388 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 195) took 1301 ms. Trace:
W20260927 23:28:08.452417 2390 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 consensus state has no leader
W20260927 23:28:08.444597 2384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_732525debefe48c49e2657fdd56b038b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:08.456624 2390 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:51972 (request call id 197) took 1156 ms. Trace:
I20260927 23:28:08.457655 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_b397e8b13f8e47a6a305e9f36d45f986"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:08.454206 2388 rpcz_store.cc:276] 0927 23:28:07.152418 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.191081 (+1038663us) service_pool.cc:227] Handling call
0927 23:28:08.192221 (+ 1140us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:08.208865 (+ 16644us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:28:08.208989 (+ 124us) catalog_manager.cc:2761] Locking tablets
0927 23:28:08.209483 (+ 494us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:28:08.451035 (+241552us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:28:08.451230 (+ 195us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:08.451319 (+ 89us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:08.453851 (+ 2532us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:08.321321 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:08.321677 (+ 356us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:08.321699 (+ 22us) write_op.cc:435] Acquired schema lock
0927 23:28:08.321711 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:08.322003 (+ 292us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:08.322039 (+ 36us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:08.322060 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:08.322364 (+ 304us) tablet.cc:699] Row locks acquired
0927 23:28:08.322378 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:08.322790 (+ 412us) write_op.cc:270] Start()
0927 23:28:08.322874 (+ 84us) write_op.cc:276] Timestamp: P: 1790551688322748 usec, L: 0
0927 23:28:08.322891 (+ 17us) op_driver.cc:331] REPLICATION: starting
0927 23:28:08.323369 (+ 478us) log.cc:842] Serialized 722 byte log entry
0927 23:28:08.447486 (+124117us) op_driver.cc:444] REPLICATION: finished
0927 23:28:08.448005 (+ 519us) write_op.cc:301] APPLY: starting
0927 23:28:08.448071 (+ 66us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:08.448180 (+ 109us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:08.448337 (+ 157us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:08.448596 (+ 259us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:08.448733 (+ 137us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:08.448778 (+ 45us) write_op.cc:312] APPLY: finished
0927 23:28:08.448957 (+ 179us) log.cc:842] Serialized 29 byte log entry
0927 23:28:08.449114 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.449178 (+ 64us) write_op.cc:454] Released schema lock
0927 23:28:08.449208 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:08.449377 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.449400 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":374,"apply.run_cpu_time_us":1579,"apply.run_wall_time_us":1579,"mutex_wait_us":908,"num_ops":2,"prepare.queue_time_us":95903,"prepare.run_cpu_time_us":2003,"prepare.run_wall_time_us":3413,"raft.queue_time_us":5088,"raft.run_cpu_time_us":641,"raft.run_wall_time_us":642,"replication_time_us":124524}]]}
W20260927 23:28:08.459026 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b397e8b13f8e47a6a305e9f36d45f986 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:08.330742 2800 rpcz_store.cc:276] 0927 23:28:07.278033 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:07.278210 (+ 177us) service_pool.cc:227] Handling call
0927 23:28:07.402701 (+124491us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:07.402771 (+ 70us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:07.437489 (+ 34718us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.056380 (+618891us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:08.318069 (+261689us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":242423}
I20260927 23:28:08.390405 2821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.2.57.1:35131 (request call id 136) took 1289 ms. Trace:
I20260927 23:28:08.371771 3558 ts_tablet_manager.cc:1905] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:08.434513 3167 tablet_bootstrap.cc:492] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:08.461174 3167 ts_tablet_manager.cc:1397] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 1.061s user 0.011s sys 0.002s
I20260927 23:28:08.434911 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet 2411d7c53a154fafad9edda9fa75bb36 with delete_type TABLET_DATA_DELETED (replaced by tablet 73cfb41befe44e5189029abafe7e326d) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.456900 2390 rpcz_store.cc:276] 0927 23:28:07.300717 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.224230 (+923513us) service_pool.cc:227] Handling call
0927 23:28:08.239987 (+ 15757us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:08.240695 (+ 708us) catalog_manager.cc:3616] Apply alter schema
0927 23:28:08.241208 (+ 513us) catalog_manager.cc:3639] Acquired catalog manager lock
0927 23:28:08.241369 (+ 161us) catalog_manager.cc:3786] Updating metadata on disk
0927 23:28:08.451681 (+210312us) catalog_manager.cc:3817] Committing alterations to in-memory state
0927 23:28:08.456570 (+ 4889us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:08.324807 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:08.325139 (+ 332us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:08.325160 (+ 21us) write_op.cc:435] Acquired schema lock
0927 23:28:08.325173 (+ 13us) tablet.cc:660] Decoding operations
0927 23:28:08.325393 (+ 220us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:08.325425 (+ 32us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:08.325443 (+ 18us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:08.325579 (+ 136us) tablet.cc:699] Row locks acquired
0927 23:28:08.325592 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:28:08.325680 (+ 88us) write_op.cc:270] Start()
0927 23:28:08.325751 (+ 71us) write_op.cc:276] Timestamp: P: 1790551688325642 usec, L: 0
0927 23:28:08.325767 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:08.326157 (+ 390us) log.cc:842] Serialized 405 byte log entry
0927 23:28:08.447679 (+121522us) op_driver.cc:444] REPLICATION: finished
0927 23:28:08.449681 (+ 2002us) write_op.cc:301] APPLY: starting
0927 23:28:08.449727 (+ 46us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:08.449807 (+ 80us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:08.449891 (+ 84us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:08.450053 (+ 162us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:08.450165 (+ 112us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:08.450205 (+ 40us) write_op.cc:312] APPLY: finished
0927 23:28:08.450363 (+ 158us) log.cc:842] Serialized 23 byte log entry
0927 23:28:08.450513 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.450564 (+ 51us) write_op.cc:454] Released schema lock
0927 23:28:08.450590 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:08.450742 (+ 152us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.450765 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1904,"apply.run_cpu_time_us":1203,"apply.run_wall_time_us":1202,"num_ops":1,"prepare.queue_time_us":82470,"prepare.run_cpu_time_us":1571,"prepare.run_wall_time_us":3296,"raft.queue_time_us":2647,"raft.run_cpu_time_us":13,"raft.run_wall_time_us":13,"replication_time_us":121857}]]}
I20260927 23:28:08.462289 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_05ddc811340c431890c23fef6ab1ba6d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:08.463629 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_05ddc811340c431890c23fef6ab1ba6d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:08.438433 2805 tablet_service.cc:1467] Processing CreateTablet for tablet ee5d073c8df84f0884dc0f0d88eeed9f (DEFAULT_TABLE table=default.Table_92117e5358444ef7bb1d38724fd8f933 [id=7163a6fa245d4c3b85f113b2786872dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.438794 2795 tablet_service.cc:1467] Processing CreateTablet for tablet a8df6d0652634def9c8946c346dea702 (DEFAULT_TABLE table=default.Table_4dd942a54bfc41a882f7e66a38e4d1fb [id=996f8a130c7f48b4b23b74dad8f5fee4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.467620 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee5d073c8df84f0884dc0f0d88eeed9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.468235 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet cc1c0da6ea5c4a09a71555f37becae33 with delete_type TABLET_DATA_DELETED (replaced by tablet 78e099028f7847d1b3de7d437be97de0) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.468879 2806 tablet_service.cc:1514] Processing DeleteTablet for tablet a80bff324902498fa5c04fe8cbc05f9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:02 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.471714 2939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31676cfbbad04e728adb4b8aa8d905a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.476994 2390 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_92117e5358444ef7bb1d38724fd8f933" } new_table_name: "default.Table_9af374fc8a5d47a0aa54bec90ce8bc1c" modify_external_catalogs: true
I20260927 23:28:08.478721 2940 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.479710 2949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8df6d0652634def9c8946c346dea702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.480397 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 119a1338b04740d0bbf291c8fbe34992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.490348 2907 leader_election.cc:302] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:08.491454 2907 leader_election.cc:302] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:08.497375 3575 raft_consensus.cc:2781] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:08.497745 3575 raft_consensus.cc:482] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:08.498021 3575 raft_consensus.cc:3037] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:08.500486 3570 raft_consensus.cc:2781] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:08.500793 3570 raft_consensus.cc:482] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:08.501070 3570 raft_consensus.cc:3037] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:08.505254 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4277045ec344ad7a48db9b14bea5e7b" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:07.925835 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 8cd0972f635d4979abf5fd30e9b02b1d (DEFAULT_TABLE table=default.Table_813e5d484f8f462b86e9136a57eebacd [id=b215370f7b594ab28c3e2a34de2a6f16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.509542 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8cd0972f635d4979abf5fd30e9b02b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.509778 2941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db8996200d9c4c5f8572d1db2dbfe4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.514698 3072 tablet_bootstrap.cc:492] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:08.515075 3072 ts_tablet_manager.cc:1397] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.162s user 0.010s sys 0.000s
I20260927 23:28:08.518113 3072 raft_consensus.cc:348] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.519503 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 31676cfbbad04e728adb4b8aa8d905a6 (DEFAULT_TABLE table=default.Table_3f55d539f6264cef8baea2cc68a8fe37 [id=df3186e81f1a4ea0977995aeef912b77]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.520414 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31676cfbbad04e728adb4b8aa8d905a6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:08.493659 2644 consensus_meta.cc:287] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent flushing consensus metadata: real 1.138s user 0.007s sys 0.002s
I20260927 23:28:08.521142 3167 raft_consensus.cc:348] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.521770 3167 raft_consensus.cc:374] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:08.521982 3167 raft_consensus.cc:729] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:08.524438 2940 tablet_service.cc:1467] Processing CreateTablet for tablet 38be781ead0e470e9928881594b4b98f (DEFAULT_TABLE table=default.Table_d609f64deb9647f79c581917277e1f93 [id=7148bae99bd941abb55f7141b044f9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.525462 2391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 3f71f948891b4d7fa95c3a1fbd92ce1e
I20260927 23:28:08.525573 2940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38be781ead0e470e9928881594b4b98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.526692 2391 catalog_manager.cc:6565] ReplaceTablet: tablet 3f71f948891b4d7fa95c3a1fbd92ce1e deleted and replaced by tablet 0e0c12031a734ca0a6536404409c62ec
I20260927 23:28:08.526983 2391 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:51972 (request call id 194) took 1407 ms. Trace:
I20260927 23:28:08.531615 2907 leader_election.cc:302] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:08.533008 3572 raft_consensus.cc:2781] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:08.533334 3572 raft_consensus.cc:482] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:08.533618 3572 raft_consensus.cc:3037] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:08.533998 2387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet f58349edf0db40f98951e116fc2582b0
I20260927 23:28:08.535349 2387 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 198) took 1086 ms. Trace:
I20260927 23:28:08.542026 3572 raft_consensus.cc:504] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.535629 2387 rpcz_store.cc:276] 0927 23:28:07.448877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.242709 (+793832us) service_pool.cc:227] Handling call
0927 23:28:08.323035 (+ 80326us) spinlock_profiling.cc:243] Waited 79.4 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b3466 00007f8bb3bf4b34 00007f8bb3bf4ac2 00007f8bb3bf49f1 00007f8bb3bf496c 00007f8bb3bf3c22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:08.324037 (+ 1002us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:08.324762 (+ 725us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:28:08.324885 (+ 123us) catalog_manager.cc:2761] Locking tablets
0927 23:28:08.325372 (+ 487us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:28:08.533607 (+208235us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:28:08.533808 (+ 201us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:08.533914 (+ 106us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:08.535286 (+ 1372us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:08.338700 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:08.339040 (+ 340us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:08.339065 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:28:08.339090 (+ 25us) tablet.cc:660] Decoding operations
0927 23:28:08.339393 (+ 303us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:08.339430 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:08.339450 (+ 20us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:08.339708 (+ 258us) tablet.cc:699] Row locks acquired
0927 23:28:08.339720 (+ 12us) write_op.cc:260] PREPARE: finished
0927 23:28:08.339821 (+ 101us) write_op.cc:270] Start()
0927 23:28:08.339897 (+ 76us) write_op.cc:276] Timestamp: P: 1790551688339780 usec, L: 0
0927 23:28:08.339914 (+ 17us) op_driver.cc:331] REPLICATION: starting
0927 23:28:08.340373 (+ 459us) log.cc:842] Serialized 753 byte log entry
0927 23:28:08.520989 (+180616us) op_driver.cc:444] REPLICATION: finished
0927 23:28:08.523468 (+ 2479us) write_op.cc:301] APPLY: starting
0927 23:28:08.523519 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:08.523611 (+ 92us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:08.523686 (+ 75us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:08.523915 (+ 229us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:08.524034 (+ 119us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:08.524072 (+ 38us) write_op.cc:312] APPLY: finished
0927 23:28:08.524223 (+ 151us) log.cc:842] Serialized 29 byte log entry
0927 23:28:08.532972 (+ 8749us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.533038 (+ 66us) write_op.cc:454] Released schema lock
0927 23:28:08.533085 (+ 47us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:08.533254 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.533276 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":182516608,"child_traces":[["op",{"apply.queue_time_us":2371,"apply.run_cpu_time_us":1422,"apply.run_wall_time_us":9990,"num_ops":2,"prepare.queue_time_us":7473,"prepare.run_cpu_time_us":1919,"prepare.run_wall_time_us":3232,"raft.queue_time_us":73,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":11,"replication_time_us":181014}]]}
I20260927 23:28:08.512710 2821 rpcz_store.cc:276] 0927 23:28:07.084048 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:07.182596 (+ 98548us) service_pool.cc:227] Handling call
0927 23:28:07.182962 (+ 366us) raft_consensus.cc:1457] Updating replica for 1 ops
0927 23:28:07.691198 (+508236us) raft_consensus.cc:1520] Early marking committed up to index 0
0927 23:28:07.691230 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0927 23:28:07.691691 (+ 461us) log.cc:842] Serialized 26 byte log entry
0927 23:28:07.877377 (+185686us) raft_consensus.cc:1631] Marking committed up to 1
0927 23:28:07.896895 (+ 19518us) log.cc:842] Serialized 14 byte log entry
0927 23:28:07.904352 (+ 7457us) raft_consensus.cc:1683] Filling consensus response to leader.
0927 23:28:07.904431 (+ 79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0927 23:28:08.363579 (+459148us) raft_consensus.cc:1671] finished
0927 23:28:08.363596 (+ 17us) raft_consensus.cc:1677] UpdateReplicas() finished
0927 23:28:08.373572 (+ 9976us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":12610,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":29,"raft.queue_time_us":6570,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":243,"thread_start_us":185356,"threads_started":1,"wal-append.queue_time_us":184158}
I20260927 23:28:08.546818 3572 leader_election.cc:288] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:08.527294 2391 rpcz_store.cc:276] 0927 23:28:07.119251 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.175129 (+1055878us) service_pool.cc:227] Handling call
0927 23:28:08.216200 (+ 41071us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b5b81 00007f8bb3c10d37 00007f8bb3c10cc2 00007f8bb3c10bf1 00007f8bb3c10b6c 00007f8bb3c0fe22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:08.526935 (+310735us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:08.328196 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:08.328548 (+ 352us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:08.328568 (+ 20us) write_op.cc:435] Acquired schema lock
0927 23:28:08.328580 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:08.328848 (+ 268us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:08.328879 (+ 31us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:08.328898 (+ 19us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:08.329068 (+ 170us) tablet.cc:699] Row locks acquired
0927 23:28:08.329082 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:08.329171 (+ 89us) write_op.cc:270] Start()
0927 23:28:08.329242 (+ 71us) write_op.cc:276] Timestamp: P: 1790551688329134 usec, L: 0
0927 23:28:08.329257 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:08.329679 (+ 422us) log.cc:842] Serialized 619 byte log entry
0927 23:28:08.520788 (+191109us) op_driver.cc:444] REPLICATION: finished
0927 23:28:08.521801 (+ 1013us) write_op.cc:301] APPLY: starting
0927 23:28:08.521856 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:08.521956 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:08.522050 (+ 94us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:08.522355 (+ 305us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:08.522570 (+ 215us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:08.522611 (+ 41us) write_op.cc:312] APPLY: finished
0927 23:28:08.522783 (+ 172us) log.cc:842] Serialized 29 byte log entry
0927 23:28:08.522919 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.522980 (+ 61us) write_op.cc:454] Released schema lock
0927 23:28:08.523009 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:08.523176 (+ 167us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:08.523199 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":92525440,"child_traces":[["op",{"apply.queue_time_us":871,"apply.run_cpu_time_us":1558,"apply.run_wall_time_us":1558,"num_ops":2,"prepare.queue_time_us":28425,"prepare.run_cpu_time_us":1724,"prepare.run_wall_time_us":6154,"raft.queue_time_us":64,"raft.run_cpu_time_us":10,"raft.run_wall_time_us":10,"replication_time_us":191458}]]}
I20260927 23:28:08.540717 2795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8df6d0652634def9c8946c346dea702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.553663 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 38be781ead0e470e9928881594b4b98f (DEFAULT_TABLE table=default.Table_d609f64deb9647f79c581917277e1f93 [id=7148bae99bd941abb55f7141b044f9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.554579 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38be781ead0e470e9928881594b4b98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.555789 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f71f948891b4d7fa95c3a1fbd92ce1e with delete_type TABLET_DATA_DELETED (replaced by tablet 0e0c12031a734ca0a6536404409c62ec) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.556581 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f71f948891b4d7fa95c3a1fbd92ce1e with delete_type TABLET_DATA_DELETED (replaced by tablet 0e0c12031a734ca0a6536404409c62ec) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.557168 2798 tablet_service.cc:1514] Processing DeleteTablet for tablet f58349edf0db40f98951e116fc2582b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.557555 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet f58349edf0db40f98951e116fc2582b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:44952
W20260927 23:28:08.559438 2390 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ee5d073c8df84f0884dc0f0d88eeed9f consensus state has no leader
I20260927 23:28:08.459894 2801 rpcz_store.cc:276] 0927 23:28:07.170984 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:07.171162 (+ 178us) service_pool.cc:227] Handling call
0927 23:28:07.270496 (+ 99334us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:07.270602 (+ 106us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:07.992689 (+722087us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.424796 (+432107us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:08.425190 (+ 394us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:08.560868 2800 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.564651 3167 consensus_queue.cc:260] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.566805 3167 ts_tablet_manager.cc:1428] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.105s user 0.004s sys 0.000s
I20260927 23:28:08.567239 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f530db3b52944f6a283ee9e2bf66828" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:08.567636 3167 tablet_bootstrap.cc:492] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
W20260927 23:28:08.576421 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 43736us
W20260927 23:28:08.578852 2906 leader_election.cc:341] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:08.596661 3070 tablet_bootstrap.cc:654] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:08.600368 2644 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54604 (request call id 147) took 1261 ms. Trace:
I20260927 23:28:08.600857 2644 rpcz_store.cc:276] 0927 23:28:07.339395 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:07.339645 (+ 250us) service_pool.cc:227] Handling call
0927 23:28:07.341522 (+ 1877us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:07.341672 (+ 150us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:07.354916 (+ 13244us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.599609 (+1244693us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:08.600244 (+ 635us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:08.616693 2644 tablet_service.cc:1467] Processing CreateTablet for tablet 0a2f98dd905d40959dc676f94a4a595b (DEFAULT_TABLE table=default.Table_a902e331f690448a872f6c104857f2ca [id=1acae7217ece44919a8d247f5cf155d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.618052 2644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a2f98dd905d40959dc676f94a4a595b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.618499 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet 410294c28c49404e9c547b27111712eb with delete_type TABLET_DATA_DELETED (replaced by tablet 8cd0972f635d4979abf5fd30e9b02b1d) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:08.619693 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet a80bff324902498fa5c04fe8cbc05f9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:02 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:08.616962 2648 tablet_service.cc:1467] Processing CreateTablet for tablet f253a54ab3b44822aee266314fa45495 (DEFAULT_TABLE table=default.Table_3bae57755da641d491998ef6ab5dd4ec [id=7fadbf61f94e4214bea8977753469e8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.620668 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet 3acdfe7faa1a4d5d8e340f09f1cae93e with delete_type TABLET_DATA_DELETED (replaced by tablet f58349edf0db40f98951e116fc2582b0) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:08.623231 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet 04f670db808c4e289c655679da9d2d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:08.625960 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f253a54ab3b44822aee266314fa45495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.580286 3570 raft_consensus.cc:504] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.636153 3575 raft_consensus.cc:504] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.640901 3575 leader_election.cc:288] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:08.648447 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet 7767a32c07454fe6a92154429a3b6899 with delete_type TABLET_DATA_DELETED (replaced by tablet 49883297d2b046b99f2ec1343c276ff9) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:08.649585 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet a4277045ec344ad7a48db9b14bea5e7b with delete_type TABLET_DATA_DELETED (replaced by tablet 0a2f98dd905d40959dc676f94a4a595b) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:08.658833 2608 leader_election.cc:302] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:08.659842 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 171730us
I20260927 23:28:08.661432 3516 raft_consensus.cc:2781] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:08.662266 3516 raft_consensus.cc:686] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:08.662642 2386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_813e5d484f8f462b86e9136a57eebacd" } modify_external_catalogs: true
I20260927 23:28:08.663343 2386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_813e5d484f8f462b86e9136a57eebacd" } modify_external_catalogs: true
I20260927 23:28:08.663228 3516 consensus_queue.cc:237] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:08.673471 2645 consensus_meta.cc:287] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent flushing consensus metadata: real 0.751s user 0.008s sys 0.002s
I20260927 23:28:08.674695 2645 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54604 (request call id 146) took 1368 ms. Trace:
I20260927 23:28:08.675134 2645 rpcz_store.cc:276] 0927 23:28:07.305888 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:07.306135 (+ 247us) service_pool.cc:227] Handling call
0927 23:28:07.308576 (+ 2441us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:07.308662 (+ 86us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:07.889940 (+581278us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.674165 (+784225us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:08.674634 (+ 469us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:08.584321 3072 raft_consensus.cc:374] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:08.677170 3072 raft_consensus.cc:729] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:08.678007 3559 ts_tablet_manager.cc:1918] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:08.677862 3072 consensus_queue.cc:260] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.680111 3072 ts_tablet_manager.cc:1428] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.165s user 0.006s sys 0.001s
I20260927 23:28:08.681981 2385 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_732525debefe48c49e2657fdd56b038b" } new_table_name: "default.Table_6487eaf832c448fbbc9b1e6622e5f105" modify_external_catalogs: true
I20260927 23:28:08.682778 2608 leader_election.cc:302] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:08.684978 3072 tablet_bootstrap.cc:492] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:08.583781 2943 tablet_service.cc:1467] Processing CreateTablet for tablet 1542b6be05634f6a88694df446373fe9 (DEFAULT_TABLE table=default.Table_1322a4efff174b9e8c09d2be9f8e4570 [id=01acb3e29c634be0bed0ecb718c10591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.695080 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1542b6be05634f6a88694df446373fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.698825 3336 ts_tablet_manager.cc:1918] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.3
I20260927 23:28:08.666157 3570 leader_election.cc:288] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:08.699734 3336 log.cc:1196] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/410294c28c49404e9c547b27111712eb
I20260927 23:28:08.700232 3592 raft_consensus.cc:2781] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:08.700637 3592 raft_consensus.cc:482] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:08.700935 3592 raft_consensus.cc:3037] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:08.701522 3336 ts_tablet_manager.cc:1939] T 410294c28c49404e9c547b27111712eb P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
W20260927 23:28:08.706094 2378 consensus_peers.cc:487] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f -> Peer 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571): Couldn't send request to peer 0cdd64d48a684d5ebccc2e2e8a4e3150. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260927 23:28:08.576799 2794 tablet_service.cc:1467] Processing CreateTablet for tablet 1542b6be05634f6a88694df446373fe9 (DEFAULT_TABLE table=default.Table_1322a4efff174b9e8c09d2be9f8e4570 [id=01acb3e29c634be0bed0ecb718c10591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.572269 3167 tablet_bootstrap.cc:654] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:08.672581 3577 raft_consensus.cc:482] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:08.707314 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1542b6be05634f6a88694df446373fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.707103 3577 raft_consensus.cc:504] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:08.666646 2823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3200d6bd5cb24ff5b1bbd59af9353dbe" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:08.708938 3577 leader_election.cc:288] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:08.608920 2763 leader_election.cc:302] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:08.639139 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e596517324300be30df8506a18904" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:08.710196 2763 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 101364us
I20260927 23:28:08.711534 3072 tablet_bootstrap.cc:654] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:08.712201 2906 leader_election.cc:341] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:08.712872 2906 leader_election.cc:341] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:08.717728 2761 leader_election.cc:341] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: 9cce47b050b445dfba7be5ea3b6a8636
I20260927 23:28:08.725289 2388 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:08.727918 2386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 8cd0972f635d4979abf5fd30e9b02b1d
I20260927 23:28:08.733181 2796 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cd0972f635d4979abf5fd30e9b02b1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.679013 3559 log.cc:1196] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/cc1c0da6ea5c4a09a71555f37becae33
W20260927 23:28:08.740833 2385 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e795a6d7eee140b181c2846ee5c43e24 has no consensus state
I20260927 23:28:08.757632 3559 ts_tablet_manager.cc:1939] T cc1c0da6ea5c4a09a71555f37becae33 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:08.758481 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 134) took 2094 ms. Trace:
I20260927 23:28:08.758881 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 97) took 4264 ms. Trace:
I20260927 23:28:08.760193 3167 tablet_bootstrap.cc:492] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:08.760687 3167 ts_tablet_manager.cc:1397] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.193s user 0.006s sys 0.003s
I20260927 23:28:08.762586 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_646f1aa916434287ab21321062e1a5d1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:08.765056 3558 ts_tablet_manager.cc:1918] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:08.765573 3558 log.cc:1196] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/2d837daee7a04b338813519acf53dfae
I20260927 23:28:08.766418 3558 ts_tablet_manager.cc:1939] T 2d837daee7a04b338813519acf53dfae P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:08.766577 2390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_e4b17b19ddc8470a94bb58e6ac8e6750"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:08.768723 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 112) took 1915 ms. Trace:
W20260927 23:28:08.768050 2390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e4b17b19ddc8470a94bb58e6ac8e6750 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:08.770431 2945 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cd0972f635d4979abf5fd30e9b02b1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.768708 2384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_96276d84ee11493d89908e4ff16dc7fe" } modify_external_catalogs: true
I20260927 23:28:08.772593 2384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_96276d84ee11493d89908e4ff16dc7fe" } modify_external_catalogs: true
W20260927 23:28:08.773129 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_646f1aa916434287ab21321062e1a5d1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:08.769402 2392 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_05ddc811340c431890c23fef6ab1ba6d" } new_table_name: "default.Table_d0bc6f0546da41d1b7ce9d874ad0b416" modify_external_catalogs: true
I20260927 23:28:08.784761 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_aea0f7ce6a4f48e6a3c1f1e349d6ffb7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:08.759238 3336 rpcz_store.cc:276] 0927 23:28:04.494159 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:04.505755 (+ 11596us) service_pool.cc:227] Handling call
0927 23:28:08.382278 (+3876523us) ts_tablet_manager.cc:1145] Deleting tablet 410294c28c49404e9c547b27111712eb
0927 23:28:08.382325 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:08.476629 (+ 94304us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.698286 (+221657us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.758794 (+ 60508us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3869737}
W20260927 23:28:08.786171 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_aea0f7ce6a4f48e6a3c1f1e349d6ffb7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:08.786540 3072 tablet_bootstrap.cc:492] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:08.786912 3072 ts_tablet_manager.cc:1397] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.102s user 0.007s sys 0.002s
I20260927 23:28:08.789124 3336 tablet_replica.cc:330] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:08.772812 2796 tablet_service.cc:1514] Processing DeleteTablet for tablet a4277045ec344ad7a48db9b14bea5e7b with delete_type TABLET_DATA_DELETED (replaced by tablet 0a2f98dd905d40959dc676f94a4a595b) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.792018 3336 raft_consensus.cc:2229] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:08.792220 3558 rpcz_store.cc:276] 0927 23:28:06.852980 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.853272 (+ 292us) service_pool.cc:227] Handling call
0927 23:28:08.060636 (+1207364us) ts_tablet_manager.cc:1145] Deleting tablet 2d837daee7a04b338813519acf53dfae
0927 23:28:08.060673 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:08.587128 (+526455us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.719168 (+132040us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.768666 (+ 49498us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3768,"tablet-delete.queue_time_us":1191857}
I20260927 23:28:08.793617 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 2d837daee7a04b338813519acf53dfae (table default.Table_08f6c5348e2b41feba84b48a70ac7a8c [id=5e610846ba4f4cd7907236b723747067]) successfully deleted
I20260927 23:28:08.794502 3558 tablet_replica.cc:330] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:08.795270 3558 raft_consensus.cc:2229] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:08.795763 3558 raft_consensus.cc:2258] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:08.796224 2384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet cfbbe17f54d1418b95efc3943db0b6c1
I20260927 23:28:08.788980 3167 raft_consensus.cc:348] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.796838 3167 raft_consensus.cc:374] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:08.797093 3167 raft_consensus.cc:729] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:08.797646 3167 consensus_queue.cc:260] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.800680 2796 tablet_service.cc:1514] Processing DeleteTablet for tablet cfbbe17f54d1418b95efc3943db0b6c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.787052 2795 tablet_service.cc:1514] Processing DeleteTablet for tablet 04f670db808c4e289c655679da9d2d9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:08.759091 3559 rpcz_store.cc:276] 0927 23:28:06.663830 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.664005 (+ 175us) service_pool.cc:227] Handling call
0927 23:28:06.737924 (+ 73919us) ts_tablet_manager.cc:1145] Deleting tablet cc1c0da6ea5c4a09a71555f37becae33
0927 23:28:06.737982 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:07.969281 (+1231299us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.677570 (+708289us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.758417 (+ 80847us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":88827,"tablet-delete.queue_time_us":73253,"thread_start_us":72857,"threads_started":1}
I20260927 23:28:08.804338 3559 tablet_replica.cc:330] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:08.811128 3623 raft_consensus.cc:482] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:08.811671 3623 raft_consensus.cc:504] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.813719 3623 leader_election.cc:288] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:08.814131 3072 raft_consensus.cc:348] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.814782 3072 raft_consensus.cc:374] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:08.815004 3072 raft_consensus.cc:729] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:08.815620 2949 tablet_service.cc:1514] Processing DeleteTablet for tablet cfbbe17f54d1418b95efc3943db0b6c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:08.816269 3072 consensus_queue.cc:260] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.817044 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1912a82ff46f4f9aaa24b7621cc3995d" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:08.818104 3072 ts_tablet_manager.cc:1428] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.031s user 0.002s sys 0.003s
I20260927 23:28:08.819324 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_4c3e33fd8fa1454f89c1a524de0bcc39"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:08.819721 2906 leader_election.cc:341] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:08.824801 2392 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e81df7c7f1a04eb9904dd2a1a5a596c4 has no consensus state
W20260927 23:28:08.834440 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4c3e33fd8fa1454f89c1a524de0bcc39 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:08.837031 3336 raft_consensus.cc:2258] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:08.839012 3336 ts_tablet_manager.cc:1905] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:08.841933 3592 raft_consensus.cc:504] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:08.843479 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 164548us
I20260927 23:28:08.845671 3250 consensus_queue.cc:1035] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:08.846387 2650 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54604 (request call id 150) took 1143 ms. Trace:
I20260927 23:28:08.846735 2650 rpcz_store.cc:276] 0927 23:28:07.703105 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:07.925158 (+222053us) service_pool.cc:227] Handling call
0927 23:28:08.508760 (+583602us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:08.508929 (+ 169us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:08.736457 (+227528us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.845884 (+109427us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:08.846321 (+ 437us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:08.848604 3540 consensus_queue.cc:1035] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:08.851131 3250 consensus_queue.cc:1035] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:08.858963 3558 ts_tablet_manager.cc:1905] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:08.864543 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:08.825568 2609 leader_election.cc:341] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:08.872793 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 47349.5us
I20260927 23:28:08.873924 3559 raft_consensus.cc:2229] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:08.873952 3250 consensus_queue.cc:1035] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:08.874780 3559 raft_consensus.cc:2258] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:08.876499 2608 leader_election.cc:302] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:08.818892 3072 tablet_bootstrap.cc:492] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:08.892999 3619 raft_consensus.cc:2781] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:08.893471 3619 raft_consensus.cc:482] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:08.893819 3619 raft_consensus.cc:3037] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:08.898811 3167 ts_tablet_manager.cc:1428] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.138s user 0.004s sys 0.007s
I20260927 23:28:08.907014 3167 tablet_bootstrap.cc:492] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:08.908659 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet ac19099dd7bf4b669d504a654ee69482 with delete_type TABLET_DATA_DELETED (replaced by tablet 9eb27269a590476f878ee52a2aefd141) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:08.910796 3070 tablet_bootstrap.cc:492] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:08.916700 3336 ts_tablet_manager.cc:1918] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:08.917119 3336 log.cc:1196] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/3200d6bd5cb24ff5b1bbd59af9353dbe
I20260927 23:28:08.919811 3336 ts_tablet_manager.cc:1939] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
W20260927 23:28:08.920676 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 74371.9us
I20260927 23:28:08.922513 3635 raft_consensus.cc:482] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:08.922972 3635 raft_consensus.cc:504] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:08.924818 3577 raft_consensus.cc:2726] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:28:08.925582 2763 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 213909us
I20260927 23:28:08.933238 2647 tablet_service.cc:1467] Processing CreateTablet for tablet f58349edf0db40f98951e116fc2582b0 (DEFAULT_TABLE table=default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.934551 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f58349edf0db40f98951e116fc2582b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:08.937135 3070 ts_tablet_manager.cc:1397] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.951s user 0.024s sys 0.006s
I20260927 23:28:08.937665 3577 raft_consensus.cc:482] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:08.938060 3577 raft_consensus.cc:504] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:08.939879 3577 leader_election.cc:288] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:08.940979 2972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47c2f22abbe40e7ba8ea945ab96aeaf" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:08.941735 2972 raft_consensus.cc:2379] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:08.950803 2650 tablet_service.cc:1467] Processing CreateTablet for tablet db8996200d9c4c5f8572d1db2dbfe4e3 (DEFAULT_TABLE table=default.Table_511501de200744a59414f1dd8bf6b2fa [id=1d8e474b764741da8c4692ab994e26b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.951700 2390 master_service.cc:946] ReplaceTablet: received request to replace tablet fd0115095a82489a8aeb941efcc8a979 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:08.952615 2385 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_08f6c5348e2b41feba84b48a70ac7a8c" } new_table_name: "default.Table_5d215a67dc7943088d8ccb5a4691e8b6" modify_external_catalogs: true
I20260927 23:28:08.952870 3559 ts_tablet_manager.cc:1905] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:08.952811 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 119a1338b04740d0bbf291c8fbe34992 (DEFAULT_TABLE table=default.Table_91684f2e139444c0a8175fb9d1998733 [id=9677fe3601944b80ad51f032eae8069e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.949967 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 102) took 4299 ms. Trace:
I20260927 23:28:08.953807 3336 rpcz_store.cc:276] 0927 23:28:04.650976 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:04.651106 (+ 130us) service_pool.cc:227] Handling call
0927 23:28:08.788948 (+4137842us) ts_tablet_manager.cc:1145] Deleting tablet 3200d6bd5cb24ff5b1bbd59af9353dbe
0927 23:28:08.788998 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:08.867166 (+ 78168us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.902787 (+ 35621us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.949906 (+ 47119us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":44463,"tablet-delete.queue_time_us":4137331}
I20260927 23:28:08.954864 3336 tablet_replica.cc:330] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:08.919566 3072 tablet_bootstrap.cc:654] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:08.956079 2392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_79ed7e54234b47fc8f068e605aa6121a" } modify_external_catalogs: true
I20260927 23:28:08.956617 2392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_79ed7e54234b47fc8f068e605aa6121a" } modify_external_catalogs: true
I20260927 23:28:08.957527 2645 tablet_service.cc:1514] Processing DeleteTablet for tablet 2411d7c53a154fafad9edda9fa75bb36 with delete_type TABLET_DATA_DELETED (replaced by tablet 73cfb41befe44e5189029abafe7e326d) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:08.962088 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 3200d6bd5cb24ff5b1bbd59af9353dbe (table default.Table_e57880e3ffa64d3b876813b8c2246002 [id=805a37d784f64efe9c46c985e5c853f1]) successfully deleted
I20260927 23:28:08.956173 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3200d6bd5cb24ff5b1bbd59af9353dbe" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:08.966552 2652 tablet_service.cc:1467] Processing CreateTablet for tablet ee5d073c8df84f0884dc0f0d88eeed9f (DEFAULT_TABLE table=default.Table_92117e5358444ef7bb1d38724fd8f933 [id=7163a6fa245d4c3b85f113b2786872dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:08.969143 2645 tablet_service.cc:1467] Processing CreateTablet for tablet a8df6d0652634def9c8946c346dea702 (DEFAULT_TABLE table=default.Table_4dd942a54bfc41a882f7e66a38e4d1fb [id=996f8a130c7f48b4b23b74dad8f5fee4]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:08.970103 2609 leader_election.cc:341] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:08.978338 3635 leader_election.cc:288] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:08.980911 3336 raft_consensus.cc:2229] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:08.981946 3336 raft_consensus.cc:2258] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:08.983302 2763 leader_election.cc:341] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 9cce47b050b445dfba7be5ea3b6a8636
I20260927 23:28:08.983880 2763 leader_election.cc:302] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:08.986248 3336 ts_tablet_manager.cc:1905] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:08.987639 3636 raft_consensus.cc:2726] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:08.994052 3623 raft_consensus.cc:482] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:08.955651 2826 raft_consensus.cc:3037] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:08.994521 3623 raft_consensus.cc:504] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:08.994845 2761 leader_election.cc:341] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: 66f2842218fc42e5b1b83314e5f1a158
I20260927 23:28:08.960098 3167 tablet_bootstrap.cc:654] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:08.997596 3623 leader_election.cc:288] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:08.998076 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b7eb04f8ff4726b4e9fe3d048ddafc" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:08.999496 2906 leader_election.cc:341] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:09.000595 2385 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9af9b87abf1f4f78b3140a91f2525066 has no consensus state
I20260927 23:28:09.003548 2385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_49c3f2d8a1b14c02878df5f1934bbaba" } modify_external_catalogs: true
W20260927 23:28:09.004281 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 73127.1us
I20260927 23:28:09.004115 2385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_49c3f2d8a1b14c02878df5f1934bbaba" } modify_external_catalogs: true
I20260927 23:28:08.928824 3592 leader_election.cc:288] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:09.009326 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d47c2f22abbe40e7ba8ea945ab96aeaf" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:09.011039 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 31676cfbbad04e728adb4b8aa8d905a6 (DEFAULT_TABLE table=default.Table_3f55d539f6264cef8baea2cc68a8fe37 [id=df3186e81f1a4ea0977995aeef912b77]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:09.012378 2763 leader_election.cc:341] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 66f2842218fc42e5b1b83314e5f1a158
I20260927 23:28:09.012950 2763 leader_election.cc:302] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:09.013425 2651 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:09.014829 2763 leader_election.cc:302] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:09.015121 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 38be781ead0e470e9928881594b4b98f (DEFAULT_TABLE table=default.Table_d609f64deb9647f79c581917277e1f93 [id=7148bae99bd941abb55f7141b044f9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.016043 3635 raft_consensus.cc:2726] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:09.019312 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_f7dab7db47874af6b9466b544d9b8465"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:09.021426 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f7dab7db47874af6b9466b544d9b8465 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:09.027526 3558 ts_tablet_manager.cc:1918] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:09.027925 3558 log.cc:1196] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/4929fc9e396e48a189c94586d58aa55f
I20260927 23:28:09.029549 2389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_d0b2f5d4356249cd9d26d96db40990ac" } new_table_name: "default.Table_eaa0db537a9f4c38b1c0db99654a6292" modify_external_catalogs: true
I20260927 23:28:09.031038 2608 leader_election.cc:302] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:09.032804 2653 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f71f948891b4d7fa95c3a1fbd92ce1e with delete_type TABLET_DATA_DELETED (replaced by tablet 0e0c12031a734ca0a6536404409c62ec) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:09.037501 2384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_86d6643b68c849aea65e03a914f1d16f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:09.038970 2384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86d6643b68c849aea65e03a914f1d16f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:09.054143 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db8996200d9c4c5f8572d1db2dbfe4e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.059928 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee5d073c8df84f0884dc0f0d88eeed9f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:09.060552 2389 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc has no consensus state
I20260927 23:28:09.060683 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 119a1338b04740d0bbf291c8fbe34992. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.061056 3539 raft_consensus.cc:2781] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:09.061376 3539 raft_consensus.cc:482] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:09.061646 3539 raft_consensus.cc:3037] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:09.082445 3621 raft_consensus.cc:482] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:09.082897 3621 raft_consensus.cc:504] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.084246 2645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8df6d0652634def9c8946c346dea702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.087221 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31676cfbbad04e728adb4b8aa8d905a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.087239 3621 leader_election.cc:288] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:09.087764 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 56807.7us
I20260927 23:28:09.064211 3167 tablet_bootstrap.cc:492] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:09.093488 3558 ts_tablet_manager.cc:1939] T 4929fc9e396e48a189c94586d58aa55f P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:09.093443 3167 ts_tablet_manager.cc:1397] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.187s user 0.005s sys 0.005s
I20260927 23:28:09.096504 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_e6e912fbd17e426b9eb07d4ea695a57a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:09.097896 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_e6e912fbd17e426b9eb07d4ea695a57a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:09.099009 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 113) took 2142 ms. Trace:
I20260927 23:28:09.099395 3558 rpcz_store.cc:276] 0927 23:28:06.957124 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.957287 (+ 163us) service_pool.cc:227] Handling call
0927 23:28:08.794343 (+1837056us) ts_tablet_manager.cc:1145] Deleting tablet 4929fc9e396e48a189c94586d58aa55f
0927 23:28:08.794387 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:08.927932 (+133545us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:09.027305 (+ 99373us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:09.098945 (+ 71640us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3168,"tablet-delete.queue_time_us":1836374}
I20260927 23:28:09.088898 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2a39980f0c46c68becefacad890ab7" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:09.101662 2906 leader_election.cc:341] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:09.101810 3167 raft_consensus.cc:348] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.104542 3167 raft_consensus.cc:374] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:09.104769 3167 raft_consensus.cc:729] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:09.106344 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38be781ead0e470e9928881594b4b98f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.122864 2653 tablet_service.cc:1467] Processing CreateTablet for tablet 1542b6be05634f6a88694df446373fe9 (DEFAULT_TABLE table=default.Table_1322a4efff174b9e8c09d2be9f8e4570 [id=01acb3e29c634be0bed0ecb718c10591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.113072 2608 leader_election.cc:302] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:09.123292 2654 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cd0972f635d4979abf5fd30e9b02b1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:09.136469 3070 raft_consensus.cc:348] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.137195 3070 raft_consensus.cc:374] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:09.137478 3070 raft_consensus.cc:729] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:09.138077 3070 consensus_queue.cc:260] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.139535 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1542b6be05634f6a88694df446373fe9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.141611 2654 tablet_service.cc:1514] Processing DeleteTablet for tablet cfbbe17f54d1418b95efc3943db0b6c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:09.142951 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet cc1c0da6ea5c4a09a71555f37becae33 (table default.Table_29404653bfbb4e5d8192bcb604084df0 [id=baad95258e1a4358a58abc3fc1cc3737]) successfully deleted
W20260927 23:28:03.890671 3373 debug-util.cc:398] Leaking SignalData structure 0x7b0800014860 after lost signal to thread 3298
W20260927 23:28:09.149786 3373 kernel_stack_watchdog.cc:198] Thread 3298 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 379ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260927 23:28:09.141993 3378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:09.150668 3558 tablet_replica.cc:330] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:09.151343 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 4929fc9e396e48a189c94586d58aa55f (table default.Table_db0f806a353a4dbdb9d64635383bc826 [id=f0d0efa2f91248b784bb2ab87b87fbf1]) successfully deleted
W20260927 23:28:09.151513 3298 thread.cc:636] OpenStack (cloud detector) Time spent creating pthread: real 6.642s user 0.355s sys 0.580s
I20260927 23:28:09.151633 3558 raft_consensus.cc:2229] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260927 23:28:09.151813 3298 thread.cc:602] OpenStack (cloud detector) Time spent starting thread: real 6.642s user 0.355s sys 0.580s
I20260927 23:28:09.152187 3298 server_base.cc:1034] running on GCE node
W20260927 23:28:02.522008 3375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:09.157631 3298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:28:09.156800 3648 tablet.cc:1722] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:09.161643 2384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_ccc7b44c45f24b7c9b58a89e4b99f139"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:09.163216 2384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_ccc7b44c45f24b7c9b58a89e4b99f139 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:09.165529 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 52564.3us
I20260927 23:28:09.032047 3619 raft_consensus.cc:504] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.168568 3619 leader_election.cc:288] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:09.121994 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet f58349edf0db40f98951e116fc2582b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:09.172456 3635 raft_consensus.cc:2726] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:09.173990 3250 raft_consensus.cc:2781] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:09.105342 3167 consensus_queue.cc:260] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.199330 3336 ts_tablet_manager.cc:1918] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:09.202569 3167 ts_tablet_manager.cc:1428] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.108s user 0.005s sys 0.000s
I20260927 23:28:09.152181 3558 raft_consensus.cc:2258] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:09.208485 2384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_0c873b78624f41d18d0d82e10a46d738" } new_table_name: "default.Table_41e19fe9cdcf42de870775c087f52478" modify_external_catalogs: true
I20260927 23:28:09.212913 3336 log.cc:1196] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/a4277045ec344ad7a48db9b14bea5e7b
I20260927 23:28:09.213642 3336 ts_tablet_manager.cc:1939] T a4277045ec344ad7a48db9b14bea5e7b P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:09.223039 3558 ts_tablet_manager.cc:1905] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:09.240685 3167 tablet_bootstrap.cc:492] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:09.248980 2384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_59054c732de9495093fd5a97281b4cf5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:09.204459 3072 tablet_bootstrap.cc:492] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:09.264093 3072 ts_tablet_manager.cc:1397] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.445s user 0.019s sys 0.003s
I20260927 23:28:09.264119 2388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_1cf287cdda1046558956f74f784c2536" } new_table_name: "default.Table_9f833d9437504850b434e8c3ceb6662a" modify_external_catalogs: true
I20260927 23:28:09.266579 3072 raft_consensus.cc:348] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.267300 3072 raft_consensus.cc:374] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:09.267539 3072 raft_consensus.cc:729] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:09.261994 3167 tablet_bootstrap.cc:654] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:09.281688 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_6ff66acddae54d28b49ddde6a443bf7b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:09.296201 2388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 9eb27269a590476f878ee52a2aefd141 consensus state has no leader
I20260927 23:28:09.316022 2393 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98] alter complete (version 1)
W20260927 23:28:09.318821 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6ff66acddae54d28b49ddde6a443bf7b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:09.320223 3635 raft_consensus.cc:482] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:09.320696 3635 raft_consensus.cc:504] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } }
I20260927 23:28:09.322527 3635 leader_election.cc:288] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:09.325109 2761 leader_election.cc:341] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: 5efbd0211cf3408cb5ea9640d8da6694
W20260927 23:28:09.325929 2380 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet e795a6d7eee140b181c2846ee5c43e24 (table default.Table_6487eaf832c448fbbc9b1e6622e5f105 [id=c702864ac8374edbb294607cb908c17f], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet e795a6d7eee140b181c2846ee5c43e24 has no consensus state
W20260927 23:28:09.332448 2384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_59054c732de9495093fd5a97281b4cf5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:09.334138 3070 ts_tablet_manager.cc:1428] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.397s user 0.008s sys 0.000s
I20260927 23:28:09.337594 3167 tablet_bootstrap.cc:492] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:09.338068 3167 ts_tablet_manager.cc:1397] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.098s user 0.008s sys 0.003s
I20260927 23:28:09.354619 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_df694a00f8a8437cb1d2d2d01a107afc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:09.356215 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_df694a00f8a8437cb1d2d2d01a107afc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:09.356470 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "702ebdeb7c5a4b7d900dbb00236cb8ae" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:09.358029 2672 raft_consensus.cc:2454] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
W20260927 23:28:09.333017 2644 consensus_meta.cc:287] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent flushing consensus metadata: real 0.693s user 0.006s sys 0.004s
I20260927 23:28:09.371477 3070 tablet_bootstrap.cc:492] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:09.372561 3167 raft_consensus.cc:348] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.373340 3167 raft_consensus.cc:374] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:09.373582 3167 raft_consensus.cc:729] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:09.373667 2668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e596517324300be30df8506a18904" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:09.372704 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f530db3b52944f6a283ee9e2bf66828" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:09.378314 2670 raft_consensus.cc:3037] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:09.380653 3167 consensus_queue.cc:260] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.393785 3298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:28:09.399729 3298 hybrid_clock.cc:648] HybridClock initialized: now 1790551689399691 us; error 71 us; skew 500 ppm
I20260927 23:28:09.401619 3655 tablet.cc:1722] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:09.403002 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3200d6bd5cb24ff5b1bbd59af9353dbe" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:09.403781 2673 raft_consensus.cc:2379] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:09.408850 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_2bf48d7e1578434b879f5c977451801f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:09.412180 2794 tablet_service.cc:1514] Processing DeleteTablet for tablet ac19099dd7bf4b669d504a654ee69482 with delete_type TABLET_DATA_DELETED (replaced by tablet 9eb27269a590476f878ee52a2aefd141) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:09.413159 3558 ts_tablet_manager.cc:1918] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:09.416528 3558 log.cc:1196] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/9cce47b050b445dfba7be5ea3b6a8636
I20260927 23:28:09.417382 3558 ts_tablet_manager.cc:1939] T 9cce47b050b445dfba7be5ea3b6a8636 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
W20260927 23:28:09.413341 2763 leader_election.cc:341] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 5efbd0211cf3408cb5ea9640d8da6694
I20260927 23:28:09.418233 2763 leader_election.cc:302] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:09.419669 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 133) took 1065 ms. Trace:
I20260927 23:28:09.425491 3298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:28:09.422379 3635 raft_consensus.cc:2726] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:09.414103 3167 ts_tablet_manager.cc:1428] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.076s user 0.008s sys 0.000s
I20260927 23:28:09.425913 3558 rpcz_store.cc:276] 0927 23:28:08.354108 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.354200 (+ 92us) service_pool.cc:227] Handling call
0927 23:28:09.150450 (+796250us) ts_tablet_manager.cc:1145] Deleting tablet 9cce47b050b445dfba7be5ea3b6a8636
0927 23:28:09.150506 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:09.291503 (+140997us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:09.412870 (+121367us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:09.419586 (+ 6716us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":795733}
I20260927 23:28:09.424194 3070 tablet_bootstrap.cc:654] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:09.431102 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 9cce47b050b445dfba7be5ea3b6a8636 (table default.Table_92a19ba603ef488c972786364cf49c70 [id=b12aae65f933476ab57974d4b328a266]) successfully deleted
I20260927 23:28:09.376157 3649 tablet.cc:1722] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:09.434142 3558 tablet_replica.cc:330] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:09.434808 3072 consensus_queue.cc:260] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.436985 3558 raft_consensus.cc:2229] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:09.437659 3072 ts_tablet_manager.cc:1428] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.173s user 0.005s sys 0.000s
I20260927 23:28:09.438481 3072 tablet_bootstrap.cc:492] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:09.418476 3539 raft_consensus.cc:504] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.443424 2388 master_service.cc:946] ReplaceTablet: received request to replace tablet 04f670db808c4e289c655679da9d2d9d from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:09.444661 3558 raft_consensus.cc:2258] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:09.447680 2388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_86ee4c8bf9ad4144bbddded60848863f" } new_table_name: "default.Table_4892751eb5c048c584557cd3fe450453" modify_external_catalogs: true
I20260927 23:28:09.448185 3167 tablet_bootstrap.cc:492] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:09.453927 3167 tablet_bootstrap.cc:654] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:09.425640 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_2bf48d7e1578434b879f5c977451801f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:09.455677 2668 raft_consensus.cc:2379] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:09.457095 3539 leader_election.cc:288] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:09.457266 3559 ts_tablet_manager.cc:1918] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:09.460062 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1912a82ff46f4f9aaa24b7621cc3995d" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:09.461778 2670 raft_consensus.cc:2454] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:09.457880 3559 log.cc:1196] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/2d837daee7a04b338813519acf53dfae
W20260927 23:28:09.476897 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 318542us
I20260927 23:28:09.501875 2384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_aea0f7ce6a4f48e6a3c1f1e349d6ffb7" } new_table_name: "default.Table_2ed30e794ca8476ea9111e99393a0371" modify_external_catalogs: true
W20260927 23:28:09.506017 2388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 451e4549cfab40449bc7518ea86cd7c1 has no consensus state
I20260927 23:28:09.513830 2386 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:09.454351 3250 raft_consensus.cc:686] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:09.521391 2389 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet e104d49543934e5db3b188e1db065cdc (table default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757]). Expected version 1 got 0
I20260927 23:28:09.516925 3559 ts_tablet_manager.cc:1939] T 2d837daee7a04b338813519acf53dfae P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:09.522065 2389 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 1 got 0
W20260927 23:28:09.523703 2386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 24) took 774 ms (client timeout 1000 ms). Trace:
I20260927 23:28:09.526520 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2a39980f0c46c68becefacad890ab7" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:09.530994 3072 tablet_bootstrap.cc:654] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:09.532469 2795 tablet_service.cc:1467] Processing CreateTablet for tablet 972763afd57847269fafadbdd2a5f3d6 (DEFAULT_TABLE table=default.Table_b397e8b13f8e47a6a305e9f36d45f986 [id=35fe7c9553e443578f5eeff336bd55dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.527230 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 105) took 4645 ms. Trace:
I20260927 23:28:09.533661 2795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 972763afd57847269fafadbdd2a5f3d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.524416 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b7eb04f8ff4726b4e9fe3d048ddafc" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:09.469501 3558 ts_tablet_manager.cc:1905] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:09.486698 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac19099dd7bf4b669d504a654ee69482" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:09.540750 2826 raft_consensus.cc:3037] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:09.541287 2386 rpcz_store.cc:269] 0927 23:28:08.749795 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.750034 (+ 239us) service_pool.cc:227] Handling call
0927 23:28:09.523635 (+773601us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":3}
I20260927 23:28:09.486799 2823 raft_consensus.cc:3037] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:09.487393 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e596517324300be30df8506a18904" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:09.545653 2653 tablet_service.cc:1467] Processing CreateTablet for tablet e795a6d7eee140b181c2846ee5c43e24 (DEFAULT_TABLE table=default.Table_6487eaf832c448fbbc9b1e6622e5f105 [id=c702864ac8374edbb294607cb908c17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.547899 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e795a6d7eee140b181c2846ee5c43e24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.524149 2794 tablet_service.cc:1467] Processing CreateTablet for tablet e795a6d7eee140b181c2846ee5c43e24 (DEFAULT_TABLE table=default.Table_6487eaf832c448fbbc9b1e6622e5f105 [id=c702864ac8374edbb294607cb908c17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.550060 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e795a6d7eee140b181c2846ee5c43e24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.550525 3250 consensus_queue.cc:237] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:09.557453 2609 leader_election.cc:341] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:09.562018 3648 tablet.cc:1722] T 84d37dba66b84d05abf675d3eaf39a30 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:09.567512 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 135) took 2721 ms. Trace:
I20260927 23:28:09.567888 3559 rpcz_store.cc:276] 0927 23:28:06.846040 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.846154 (+ 114us) service_pool.cc:227] Handling call
0927 23:28:08.804149 (+1957995us) ts_tablet_manager.cc:1145] Deleting tablet 2d837daee7a04b338813519acf53dfae
0927 23:28:08.804209 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:09.163481 (+359272us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:09.456999 (+293518us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:09.567437 (+110438us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":49042,"tablet-delete.queue_time_us":1956619}
I20260927 23:28:09.569072 3559 tablet_replica.cc:330] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:09.569890 3559 raft_consensus.cc:2229] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:09.570430 3559 raft_consensus.cc:2258] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:09.571511 2390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet fd0115095a82489a8aeb941efcc8a979
I20260927 23:28:09.572780 2390 catalog_manager.cc:6565] ReplaceTablet: tablet fd0115095a82489a8aeb941efcc8a979 deleted and replaced by tablet 178503861b1648d9a931d1606d807ed8
I20260927 23:28:09.573158 3559 ts_tablet_manager.cc:1905] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:09.573571 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 2d837daee7a04b338813519acf53dfae (table default.Table_5d215a67dc7943088d8ccb5a4691e8b6 [id=5e610846ba4f4cd7907236b723747067]) successfully deleted
I20260927 23:28:09.575090 2796 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0115095a82489a8aeb941efcc8a979 with delete_type TABLET_DATA_DELETED (replaced by tablet 178503861b1648d9a931d1606d807ed8) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:09.579856 2645 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0115095a82489a8aeb941efcc8a979 with delete_type TABLET_DATA_DELETED (replaced by tablet 178503861b1648d9a931d1606d807ed8) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:09.587626 2392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 8b891bab5fb343b0bb94641aea7662f0
I20260927 23:28:09.591311 2645 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b891bab5fb343b0bb94641aea7662f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:09.597826 2796 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b891bab5fb343b0bb94641aea7662f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:09.602440 3298 webserver.cc:460] Webserver started at http://127.2.57.60:39731/ using document root <none> and password file <none>
I20260927 23:28:09.533262 3336 rpcz_store.cc:276] 0927 23:28:04.882249 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:04.882561 (+ 312us) service_pool.cc:227] Handling call
0927 23:28:08.954729 (+4072168us) ts_tablet_manager.cc:1145] Deleting tablet a4277045ec344ad7a48db9b14bea5e7b
0927 23:28:08.954765 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:09.075380 (+120615us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:09.199073 (+123693us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:09.527167 (+328094us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":25278,"tablet-delete.queue_time_us":4071707}
I20260927 23:28:09.605180 3336 tablet_replica.cc:330] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:09.605999 3336 raft_consensus.cc:2229] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:09.526780 3621 raft_consensus.cc:482] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:09.613680 3621 raft_consensus.cc:504] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.615100 2967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac19099dd7bf4b669d504a654ee69482" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:09.615368 3621 leader_election.cc:288] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:09.615720 2967 raft_consensus.cc:3037] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:09.620944 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet a4277045ec344ad7a48db9b14bea5e7b (table default.Table_a902e331f690448a872f6c104857f2ca [id=1acae7217ece44919a8d247f5cf155d1]) successfully deleted
I20260927 23:28:09.622139 2385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet a1890852164d492db3540a97847888fd
I20260927 23:28:09.628865 3336 raft_consensus.cc:2258] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:09.629504 2796 tablet_service.cc:1514] Processing DeleteTablet for tablet a1890852164d492db3540a97847888fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:09.632272 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0115095a82489a8aeb941efcc8a979" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:09.633966 2906 leader_election.cc:341] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:09.634505 2796 tablet_service.cc:1467] Processing CreateTablet for tablet 3a8dbc1165ac4f5b8ed792f6064cce6f (DEFAULT_TABLE table=default.Table_cccab8d895c740e0b88b2709527516e3 [id=d44851dff14d4e5084ae024995202140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.635641 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a8dbc1165ac4f5b8ed792f6064cce6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.636502 2939 tablet_service.cc:1514] Processing DeleteTablet for tablet a1890852164d492db3540a97847888fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:09.639091 3298 fs_manager.cc:359] Metadata directory not provided
I20260927 23:28:09.639911 2965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e596517324300be30df8506a18904" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:09.640762 2965 raft_consensus.cc:2379] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5ce671f028a34ea4b8f0520c2689c9a1 in current term 1: Already voted for candidate de216a4a8e4b4e1b857d8647a3b05f6d in this term.
I20260927 23:28:09.641209 3298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:28:09.641420 2941 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b891bab5fb343b0bb94641aea7662f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:09.642025 2949 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0115095a82489a8aeb941efcc8a979 with delete_type TABLET_DATA_DELETED (replaced by tablet 178503861b1648d9a931d1606d807ed8) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:09.641803 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_9b16095aa7824f10a46040ac08836015"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:09.642268 2609 consensus_peers.cc:487] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:09.642660 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 46002.2us
W20260927 23:28:09.643468 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b16095aa7824f10a46040ac08836015 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:09.655495 2826 raft_consensus.cc:2454] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:09.646881 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet d47c2f22abbe40e7ba8ea945ab96aeaf (table default.Table_41e19fe9cdcf42de870775c087f52478 [id=c8c0a7367a8342afb8ffc95627013a9c]). Expected version 1 got 0
I20260927 23:28:09.647869 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_f19cc1c4a83548278bc4b767ee445951"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:09.663152 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f19cc1c4a83548278bc4b767ee445951 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:09.658114 3636 raft_consensus.cc:482] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:09.659266 3167 tablet_bootstrap.cc:492] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:09.664028 3167 ts_tablet_manager.cc:1397] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.216s user 0.008s sys 0.003s
I20260927 23:28:09.663894 3636 raft_consensus.cc:504] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.666666 3636 leader_election.cc:288] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:09.666481 3167 raft_consensus.cc:348] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.667188 3167 raft_consensus.cc:374] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:09.667558 3167 raft_consensus.cc:729] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:09.668574 3167 consensus_queue.cc:260] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.670236 2609 leader_election.cc:302] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:09.671365 3167 ts_tablet_manager.cc:1428] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260927 23:28:09.674546 3539 raft_consensus.cc:2781] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:09.674655 2645 tablet_service.cc:1467] Processing CreateTablet for tablet 972763afd57847269fafadbdd2a5f3d6 (DEFAULT_TABLE table=default.Table_b397e8b13f8e47a6a305e9f36d45f986 [id=35fe7c9553e443578f5eeff336bd55dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.675088 3539 raft_consensus.cc:686] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:09.675755 2645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 972763afd57847269fafadbdd2a5f3d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.675913 3539 consensus_queue.cc:237] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.680429 2648 tablet_service.cc:1514] Processing DeleteTablet for tablet a1890852164d492db3540a97847888fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:09.678095 3635 raft_consensus.cc:482] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:09.693841 3635 raft_consensus.cc:504] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:09.685812 2907 leader_election.cc:341] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 74fc2f03baae4f70b38b8079deb8a854
I20260927 23:28:09.695847 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f71f948891b4d7fa95c3a1fbd92ce1e" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:09.695988 2608 consensus_peers.cc:487] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Couldn't send request to peer de216a4a8e4b4e1b857d8647a3b05f6d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:09.697069 2608 leader_election.cc:341] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: 3200d6bd5cb24ff5b1bbd59af9353dbe
I20260927 23:28:09.694736 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 3a8dbc1165ac4f5b8ed792f6064cce6f (DEFAULT_TABLE table=default.Table_cccab8d895c740e0b88b2709527516e3 [id=d44851dff14d4e5084ae024995202140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.698203 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a8dbc1165ac4f5b8ed792f6064cce6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.698477 3635 leader_election.cc:288] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:09.699729 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae1e85c32a94f868ef48ad471462b3e" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:09.701962 2943 tablet_service.cc:1467] Processing CreateTablet for tablet e795a6d7eee140b181c2846ee5c43e24 (DEFAULT_TABLE table=default.Table_6487eaf832c448fbbc9b1e6622e5f105 [id=c702864ac8374edbb294607cb908c17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.697434 2608 leader_election.cc:302] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:09.705600 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 4d5b59f73974430c91ba19c2ec307ea0 (DEFAULT_TABLE table=default.Table_001a61d6538b4af19e206fc70f19f8f4 [id=8a95d159b428431fbcba5e5d51f20ced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.706540 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d5b59f73974430c91ba19c2ec307ea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.706566 2608 leader_election.cc:302] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:09.707165 3516 raft_consensus.cc:2726] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260927 23:28:09.710629 3336 ts_tablet_manager.cc:1905] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:09.713483 2391 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_ad787a15ff9743029d703111e0a20bdf" } new_table_name: "default.Table_cda88f30ae9d451ba6327019efbb59cf" modify_external_catalogs: true
I20260927 23:28:09.688130 3619 consensus_queue.cc:1035] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260927 23:28:09.715219 3650 consensus_queue.cc:1035] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260927 23:28:09.712177 3516 raft_consensus.cc:2726] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260927 23:28:09.710244 2906 consensus_peers.cc:487] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4929fc9e396e48a189c94586d58aa55f. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:09.726157 2390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_73f1344829a046f29f209adc7eceaf42"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:09.727759 2390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_73f1344829a046f29f209adc7eceaf42 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:09.739717 2949 tablet_service.cc:1467] Processing CreateTablet for tablet 972763afd57847269fafadbdd2a5f3d6 (DEFAULT_TABLE table=default.Table_b397e8b13f8e47a6a305e9f36d45f986 [id=35fe7c9553e443578f5eeff336bd55dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.746836 3655 tablet.cc:1722] T 84d37dba66b84d05abf675d3eaf39a30 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:09.749078 3663 tablet.cc:1722] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:09.750983 2941 tablet_service.cc:1467] Processing CreateTablet for tablet 3a8dbc1165ac4f5b8ed792f6064cce6f (DEFAULT_TABLE table=default.Table_cccab8d895c740e0b88b2709527516e3 [id=d44851dff14d4e5084ae024995202140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.698081 2965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f71f948891b4d7fa95c3a1fbd92ce1e" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:09.698489 2907 leader_election.cc:302] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:09.755538 3671 tablet.cc:1722] T d47c2f22abbe40e7ba8ea945ab96aeaf P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260927 23:28:09.756048 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 112530us
I20260927 23:28:09.756736 2907 leader_election.cc:302] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:09.757899 3572 raft_consensus.cc:2726] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:09.766356 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0115095a82489a8aeb941efcc8a979" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:09.767410 2939 tablet_service.cc:1467] Processing CreateTablet for tablet 4d5b59f73974430c91ba19c2ec307ea0 (DEFAULT_TABLE table=default.Table_001a61d6538b4af19e206fc70f19f8f4 [id=8a95d159b428431fbcba5e5d51f20ced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.768350 3070 tablet_bootstrap.cc:492] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:09.768730 3070 ts_tablet_manager.cc:1397] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.397s user 0.012s sys 0.009s
I20260927 23:28:09.770744 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 26007ba8846f4b08b337c3ce03782f4a (DEFAULT_TABLE table=default.Table_0b85d1f76d2b4cd68123d6b031a5fa72 [id=d32fe286cfa44d838c2de052c884f150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.771970 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26007ba8846f4b08b337c3ce03782f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.773470 3649 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:54604 (request call id 153) took 1155 ms. Trace:
I20260927 23:28:09.776563 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e795a6d7eee140b181c2846ee5c43e24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.781849 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_849b837949944562bb32d87e0bafbb48"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:09.783476 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_849b837949944562bb32d87e0bafbb48 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:09.784584 2965 raft_consensus.cc:2379] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:09.762025 3572 raft_consensus.cc:2781] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:09.785694 3572 raft_consensus.cc:482] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:09.786001 3572 raft_consensus.cc:3037] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:09.789131 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 4d5b59f73974430c91ba19c2ec307ea0 (DEFAULT_TABLE table=default.Table_001a61d6538b4af19e206fc70f19f8f4 [id=8a95d159b428431fbcba5e5d51f20ced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.790299 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d5b59f73974430c91ba19c2ec307ea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.795931 2393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_9af374fc8a5d47a0aa54bec90ce8bc1c" } modify_external_catalogs: true
I20260927 23:28:09.796519 2393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_9af374fc8a5d47a0aa54bec90ce8bc1c" } modify_external_catalogs: true
I20260927 23:28:09.804356 3070 raft_consensus.cc:348] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.805111 3070 raft_consensus.cc:374] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:09.805138 2761 leader_election.cc:302] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:09.805341 3070 raft_consensus.cc:729] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:09.805907 3070 consensus_queue.cc:260] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.807384 3070 ts_tablet_manager.cc:1428] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.038s user 0.004s sys 0.000s
I20260927 23:28:09.773789 3649 rpcz_store.cc:276] 0927 23:28:08.617650 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.620459 (+ 2809us) service_pool.cc:227] Handling call
0927 23:28:09.773395 (+1152936us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:08.862185 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0927 23:28:08.863015 (+ 830us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0927 23:28:08.863050 (+ 35us) alter_schema_op.cc:65] Acquired schema lock
0927 23:28:08.863463 (+ 413us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0927 23:28:08.863576 (+ 113us) alter_schema_op.cc:126] START. Timestamp: 1790551688863398
0927 23:28:08.863610 (+ 34us) op_driver.cc:331] REPLICATION: starting
0927 23:28:08.864150 (+ 540us) log.cc:842] Serialized 173 byte log entry
0927 23:28:09.123235 (+259085us) op_driver.cc:444] REPLICATION: finished
0927 23:28:09.375269 (+252034us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0927 23:28:09.482238 (+106969us) log.cc:842] Serialized 5 byte log entry
0927 23:28:09.724498 (+242260us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:09.773147 (+ 48649us) log.cc:842] Serialized 14 byte log entry
0927 23:28:09.773233 (+ 86us) alter_schema_op.cc:71] Released schema lock
0927 23:28:09.773255 (+ 22us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":237047,"prepare.queue_time_us":240509,"prepare.run_cpu_time_us":5201,"prepare.run_wall_time_us":70670,"raft.queue_time_us":2016,"raft.run_cpu_time_us":856,"raft.run_wall_time_us":11288,"replication_time_us":259549,"thread_start_us":565786,"threads_started":3,"wal-append.queue_time_us":68702}]]}
I20260927 23:28:09.818586 3649 tablet.cc:1722] T d47c2f22abbe40e7ba8ea945ab96aeaf P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:09.821624 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 26007ba8846f4b08b337c3ce03782f4a (DEFAULT_TABLE table=default.Table_0b85d1f76d2b4cd68123d6b031a5fa72 [id=d32fe286cfa44d838c2de052c884f150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.822773 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26007ba8846f4b08b337c3ce03782f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.823427 2949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 972763afd57847269fafadbdd2a5f3d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.826087 2388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_0b85d1f76d2b4cd68123d6b031a5fa72" } modify_external_catalogs: true
I20260927 23:28:09.826614 2388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_0b85d1f76d2b4cd68123d6b031a5fa72" } modify_external_catalogs: true
I20260927 23:28:09.836221 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 0d9370feeb454ad3850e3db76ea5519e (DEFAULT_TABLE table=default.Table_8abcb4595c95442ba71077feec547d2c [id=73a2b941410945c08f8dd48117432dcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.837193 3559 ts_tablet_manager.cc:1918] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:09.837486 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d9370feeb454ad3850e3db76ea5519e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.837605 3559 log.cc:1196] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/4929fc9e396e48a189c94586d58aa55f
I20260927 23:28:09.838351 3559 ts_tablet_manager.cc:1939] T 4929fc9e396e48a189c94586d58aa55f P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:09.846468 3635 raft_consensus.cc:3037] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:09.846685 2940 tablet_service.cc:1467] Processing CreateTablet for tablet 26007ba8846f4b08b337c3ce03782f4a (DEFAULT_TABLE table=default.Table_0b85d1f76d2b4cd68123d6b031a5fa72 [id=d32fe286cfa44d838c2de052c884f150]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.865375 3070 tablet_bootstrap.cc:492] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:09.864033 3635 raft_consensus.cc:2726] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:09.866407 3167 tablet_bootstrap.cc:492] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:09.873380 2967 raft_consensus.cc:2454] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:09.873415 2647 tablet_service.cc:1467] Processing CreateTablet for tablet 0d9370feeb454ad3850e3db76ea5519e (DEFAULT_TABLE table=default.Table_8abcb4595c95442ba71077feec547d2c [id=73a2b941410945c08f8dd48117432dcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.874559 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d9370feeb454ad3850e3db76ea5519e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.875039 3070 tablet_bootstrap.cc:654] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:09.876703 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 4c834965c5034f39a3c6b85d6c0060cd (DEFAULT_TABLE table=default.Table_a368d1f8ac2544ba9c5b637c76f92176 [id=a69c98926747428c9e73a05392bceb30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.878003 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c834965c5034f39a3c6b85d6c0060cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.885123 2644 tablet_service.cc:1467] Processing CreateTablet for tablet 4c834965c5034f39a3c6b85d6c0060cd (DEFAULT_TABLE table=default.Table_a368d1f8ac2544ba9c5b637c76f92176 [id=a69c98926747428c9e73a05392bceb30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.886245 2644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c834965c5034f39a3c6b85d6c0060cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.891891 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 1625e523e1ca40b6b8324f302bb70d31 (DEFAULT_TABLE table=default.Table_70bf888024144eb0b5afd48ee484fb17 [id=971cece292fa4c40a43298d92eb56bb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.893213 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1625e523e1ca40b6b8324f302bb70d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.898272 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 761a67c27c7a49a3903ed6f2bf5b96d3 (DEFAULT_TABLE table=default.Table_b349caab5cc84842819f9a9bbfece301 [id=8ab43e9191d14f76ac0c8f79f39c9796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.899463 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 761a67c27c7a49a3903ed6f2bf5b96d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.839203 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 136) took 2789 ms. Trace:
I20260927 23:28:09.908661 3559 rpcz_store.cc:276] 0927 23:28:07.050164 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:07.061515 (+ 11351us) service_pool.cc:227] Handling call
0927 23:28:09.568892 (+2507377us) ts_tablet_manager.cc:1145] Deleting tablet 4929fc9e396e48a189c94586d58aa55f
0927 23:28:09.568953 (+ 61us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:09.767386 (+198433us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:09.836924 (+ 69538us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:09.839145 (+ 2221us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2506819}
I20260927 23:28:09.910219 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 154) took 1292 ms. Trace:
I20260927 23:28:09.910523 3559 rpcz_store.cc:276] 0927 23:28:08.618016 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.618211 (+ 195us) service_pool.cc:227] Handling call
0927 23:28:09.909959 (+1291748us) ts_tablet_manager.cc:1145] Deleting tablet 410294c28c49404e9c547b27111712eb
0927 23:28:09.910006 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:09.910169 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1290881}
I20260927 23:28:09.911635 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 4929fc9e396e48a189c94586d58aa55f (table default.Table_db0f806a353a4dbdb9d64635383bc826 [id=f0d0efa2f91248b784bb2ab87b87fbf1]) successfully deleted
I20260927 23:28:09.911994 3559 tablet_replica.cc:330] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:09.929800 2649 tablet_service.cc:1467] Processing CreateTablet for tablet 4322d56dd17347d0bcad0127f3869ae2 (DEFAULT_TABLE table=default.Table_8a68a0857d4f40a99f630c8ac395c419 [id=4b8d0f2f1dab4f52a666d4af1c1bb7f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.931061 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4322d56dd17347d0bcad0127f3869ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.934551 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0c12031a734ca0a6536404409c62ec (DEFAULT_TABLE table=default.Table_047c5f89196f446e86279aae36cf7f99 [id=2aecf7c487414df1985aa5f284224515]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.935552 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0c12031a734ca0a6536404409c62ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.953408 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ae1e85c32a94f868ef48ad471462b3e" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:09.955068 2799 tablet_service.cc:1467] Processing CreateTablet for tablet 1625e523e1ca40b6b8324f302bb70d31 (DEFAULT_TABLE table=default.Table_70bf888024144eb0b5afd48ee484fb17 [id=971cece292fa4c40a43298d92eb56bb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.957957 2800 tablet_service.cc:1467] Processing CreateTablet for tablet 761a67c27c7a49a3903ed6f2bf5b96d3 (DEFAULT_TABLE table=default.Table_b349caab5cc84842819f9a9bbfece301 [id=8ab43e9191d14f76ac0c8f79f39c9796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.959390 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 1a590434f92645cf9dffedef3e01bd65 (DEFAULT_TABLE table=default.Table_ad787a15ff9743029d703111e0a20bdf [id=5a15856fe7ef4bcdb10d76f8325c1152]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.960783 2802 tablet_service.cc:1467] Processing CreateTablet for tablet 4322d56dd17347d0bcad0127f3869ae2 (DEFAULT_TABLE table=default.Table_8a68a0857d4f40a99f630c8ac395c419 [id=4b8d0f2f1dab4f52a666d4af1c1bb7f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.962105 2798 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0c12031a734ca0a6536404409c62ec (DEFAULT_TABLE table=default.Table_047c5f89196f446e86279aae36cf7f99 [id=2aecf7c487414df1985aa5f284224515]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.966078 3572 raft_consensus.cc:504] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:09.967940 2939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d5b59f73974430c91ba19c2ec307ea0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.969645 2653 tablet_service.cc:1467] Processing CreateTablet for tablet 1a590434f92645cf9dffedef3e01bd65 (DEFAULT_TABLE table=default.Table_ad787a15ff9743029d703111e0a20bdf [id=5a15856fe7ef4bcdb10d76f8325c1152]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:09.970765 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a590434f92645cf9dffedef3e01bd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.985504 2941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a8dbc1165ac4f5b8ed792f6064cce6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.988798 3559 raft_consensus.cc:2229] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:09.989609 3559 raft_consensus.cc:2258] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:09.991573 3559 ts_tablet_manager.cc:1905] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:09.992960 2940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26007ba8846f4b08b337c3ce03782f4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:09.995839 3167 tablet_bootstrap.cc:654] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:10.034312 3681 log.cc:924] Time spent T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51: Append to log took a long time: real 0.066s user 0.000s sys 0.000s
I20260927 23:28:10.036248 2794 tablet_service.cc:1467] Processing CreateTablet for tablet 451e4549cfab40449bc7518ea86cd7c1 (DEFAULT_TABLE table=default.Table_4892751eb5c048c584557cd3fe450453 [id=21b7ecc140b841959c7c73067a78d678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.036603 2385 master_service.cc:946] ReplaceTablet: received request to replace tablet bbeeccc19a504e7984b293127766daf0 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:10.047286 2806 tablet_service.cc:1467] Processing CreateTablet for tablet 8b891bab5fb343b0bb94641aea7662f0 (DEFAULT_TABLE table=default.Table_79ed7e54234b47fc8f068e605aa6121a [id=b1bafaccca474a2c9d5a92260eab27a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.051282 2799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1625e523e1ca40b6b8324f302bb70d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.052639 2798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0c12031a734ca0a6536404409c62ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.053761 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4322d56dd17347d0bcad0127f3869ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.055488 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a590434f92645cf9dffedef3e01bd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.056808 2800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 761a67c27c7a49a3903ed6f2bf5b96d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.057981 2806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b891bab5fb343b0bb94641aea7662f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.059046 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 451e4549cfab40449bc7518ea86cd7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.005278 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 451e4549cfab40449bc7518ea86cd7c1 (DEFAULT_TABLE table=default.Table_4892751eb5c048c584557cd3fe450453 [id=21b7ecc140b841959c7c73067a78d678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.060456 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 451e4549cfab40449bc7518ea86cd7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.065264 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 8b891bab5fb343b0bb94641aea7662f0 (DEFAULT_TABLE table=default.Table_79ed7e54234b47fc8f068e605aa6121a [id=b1bafaccca474a2c9d5a92260eab27a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.066462 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b891bab5fb343b0bb94641aea7662f0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:10.069044 2391 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1a590434f92645cf9dffedef3e01bd65 consensus state has no leader
I20260927 23:28:10.077595 2644 tablet_service.cc:1467] Processing CreateTablet for tablet fc60d0c7a3794607814393f033238052 (DEFAULT_TABLE table=default.Table_da4f2d8a1534460a8d060ea730c5e3e8 [id=8684e02a2c7f4b7b9ae8a200a36129c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.090339 2644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc60d0c7a3794607814393f033238052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.102504 2796 tablet_service.cc:1467] Processing CreateTablet for tablet fc60d0c7a3794607814393f033238052 (DEFAULT_TABLE table=default.Table_da4f2d8a1534460a8d060ea730c5e3e8 [id=8684e02a2c7f4b7b9ae8a200a36129c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.103713 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc60d0c7a3794607814393f033238052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.102685 2649 tablet_service.cc:1467] Processing CreateTablet for tablet 78e099028f7847d1b3de7d437be97de0 (DEFAULT_TABLE table=default.Table_29404653bfbb4e5d8192bcb604084df0 [id=baad95258e1a4358a58abc3fc1cc3737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.113428 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78e099028f7847d1b3de7d437be97de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.112201 3336 ts_tablet_manager.cc:1918] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:10.115818 3336 log.cc:1196] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/04f670db808c4e289c655679da9d2d9d
I20260927 23:28:10.118410 2795 tablet_service.cc:1467] Processing CreateTablet for tablet 78e099028f7847d1b3de7d437be97de0 (DEFAULT_TABLE table=default.Table_29404653bfbb4e5d8192bcb604084df0 [id=baad95258e1a4358a58abc3fc1cc3737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.119592 2795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78e099028f7847d1b3de7d437be97de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.133920 3336 ts_tablet_manager.cc:1939] T 04f670db808c4e289c655679da9d2d9d P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:10.148418 3558 ts_tablet_manager.cc:1918] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:10.148931 3558 log.cc:1196] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/5efbd0211cf3408cb5ea9640d8da6694
I20260927 23:28:10.149864 3558 ts_tablet_manager.cc:1939] T 5efbd0211cf3408cb5ea9640d8da6694 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
W20260927 23:28:10.150323 2761 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 345247us
I20260927 23:28:10.152274 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 134) took 1797 ms. Trace:
I20260927 23:28:10.157281 2647 tablet_service.cc:1467] Processing CreateTablet for tablet c848ed8a221a4b799529ccc0d1830893 (DEFAULT_TABLE table=default.Table_9f0dec29a5d74ddaa48ef255c54b22cd [id=6f128a8acb5d47d69ead26501bd788bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.158277 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c848ed8a221a4b799529ccc0d1830893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.159451 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 355a04c4302441958d04cba33033076a (DEFAULT_TABLE table=default.Table_477638f0398348fdbb82d985e67e7c14 [id=67464814408f41fabc3895c474bb42f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.159942 2390 master_service.cc:946] ReplaceTablet: received request to replace tablet 3c6e596517324300be30df8506a18904 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:10.160645 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 355a04c4302441958d04cba33033076a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:10.160859 2282 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 112257us
I20260927 23:28:10.164881 3519 raft_consensus.cc:482] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:10.165313 3519 raft_consensus.cc:504] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.167266 3519 leader_election.cc:288] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:10.174904 2967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2a39980f0c46c68becefacad890ab7" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:10.176332 2967 raft_consensus.cc:2454] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
W20260927 23:28:10.203434 2378 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.60:41571, user_credentials={real_user=slave}} blocked reactor thread for 362592us
I20260927 23:28:10.221704 2945 tablet_service.cc:1467] Processing CreateTablet for tablet 0d9370feeb454ad3850e3db76ea5519e (DEFAULT_TABLE table=default.Table_8abcb4595c95442ba71077feec547d2c [id=73a2b941410945c08f8dd48117432dcf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.222060 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 1625e523e1ca40b6b8324f302bb70d31 (DEFAULT_TABLE table=default.Table_70bf888024144eb0b5afd48ee484fb17 [id=971cece292fa4c40a43298d92eb56bb9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.222779 2945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d9370feeb454ad3850e3db76ea5519e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.223017 2947 tablet_service.cc:1467] Processing CreateTablet for tablet 761a67c27c7a49a3903ed6f2bf5b96d3 (DEFAULT_TABLE table=default.Table_b349caab5cc84842819f9a9bbfece301 [id=8ab43e9191d14f76ac0c8f79f39c9796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.223308 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 1a590434f92645cf9dffedef3e01bd65 (DEFAULT_TABLE table=default.Table_ad787a15ff9743029d703111e0a20bdf [id=5a15856fe7ef4bcdb10d76f8325c1152]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.224117 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1625e523e1ca40b6b8324f302bb70d31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.224989 2951 tablet_service.cc:1467] Processing CreateTablet for tablet 4322d56dd17347d0bcad0127f3869ae2 (DEFAULT_TABLE table=default.Table_8a68a0857d4f40a99f630c8ac395c419 [id=4b8d0f2f1dab4f52a666d4af1c1bb7f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.225634 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 761a67c27c7a49a3903ed6f2bf5b96d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.225651 2948 tablet_service.cc:1467] Processing CreateTablet for tablet 451e4549cfab40449bc7518ea86cd7c1 (DEFAULT_TABLE table=default.Table_4892751eb5c048c584557cd3fe450453 [id=21b7ecc140b841959c7c73067a78d678]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.226410 2942 tablet_service.cc:1467] Processing CreateTablet for tablet 4c834965c5034f39a3c6b85d6c0060cd (DEFAULT_TABLE table=default.Table_a368d1f8ac2544ba9c5b637c76f92176 [id=a69c98926747428c9e73a05392bceb30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.226670 2948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 451e4549cfab40449bc7518ea86cd7c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.227316 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a590434f92645cf9dffedef3e01bd65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.230724 2942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c834965c5034f39a3c6b85d6c0060cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.228766 2944 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0c12031a734ca0a6536404409c62ec (DEFAULT_TABLE table=default.Table_047c5f89196f446e86279aae36cf7f99 [id=2aecf7c487414df1985aa5f284224515]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.232223 2951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4322d56dd17347d0bcad0127f3869ae2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.234694 3298 fs_manager.cc:709] Time spent opening directory manager: real 0.529s user 0.034s sys 0.031s
I20260927 23:28:10.180721 3635 raft_consensus.cc:482] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:10.235877 3635 raft_consensus.cc:504] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.238214 3635 leader_election.cc:288] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:10.239034 3558 rpcz_store.cc:276] 0927 23:28:08.354388 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.355400 (+ 1012us) service_pool.cc:227] Handling call
0927 23:28:09.433966 (+1078566us) ts_tablet_manager.cc:1145] Deleting tablet 5efbd0211cf3408cb5ea9640d8da6694
0927 23:28:09.434016 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:09.744644 (+310628us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.062683 (+318039us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.152217 (+ 89534us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1078081}
I20260927 23:28:10.240129 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbbe17f54d1418b95efc3943db0b6c1" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:10.240438 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 5efbd0211cf3408cb5ea9640d8da6694 (table default.Table_7c86d4308c504c4ba092bb6634326544 [id=0aa0174883974249b1120d0f02be108d]) successfully deleted
I20260927 23:28:10.190865 3167 tablet_bootstrap.cc:492] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:10.214184 2803 tablet_service.cc:1467] Processing CreateTablet for tablet c848ed8a221a4b799529ccc0d1830893 (DEFAULT_TABLE table=default.Table_9f0dec29a5d74ddaa48ef255c54b22cd [id=6f128a8acb5d47d69ead26501bd788bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.241523 2803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c848ed8a221a4b799529ccc0d1830893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.248551 3167 ts_tablet_manager.cc:1397] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.382s user 0.007s sys 0.005s
I20260927 23:28:10.250713 3167 raft_consensus.cc:348] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.251358 3167 raft_consensus.cc:374] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:10.251606 3167 raft_consensus.cc:729] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:10.215453 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 355a04c4302441958d04cba33033076a (DEFAULT_TABLE table=default.Table_477638f0398348fdbb82d985e67e7c14 [id=67464814408f41fabc3895c474bb42f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.252750 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 355a04c4302441958d04cba33033076a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.259521 3690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:28:10.265161 3298 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.007s sys 0.000s
I20260927 23:28:10.267239 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 106) took 5366 ms. Trace:
I20260927 23:28:10.267601 3336 rpcz_store.cc:276] 0927 23:28:04.901248 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:04.901359 (+ 111us) service_pool.cc:227] Handling call
0927 23:28:09.605014 (+4703655us) ts_tablet_manager.cc:1145] Deleting tablet 04f670db808c4e289c655679da9d2d9d
0927 23:28:09.605070 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:09.919883 (+314813us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.111904 (+192021us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.267156 (+155252us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22365,"tablet-delete.queue_time_us":4703121}
I20260927 23:28:10.265673 3298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/wal
uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150"
format_stamp: "Formatted at 2026-09-27 23:27:46 on dist-test-slave-jbx2"
I20260927 23:28:10.270236 3298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:28:10.274130 2608 leader_election.cc:302] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:10.284895 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 135) took 1930 ms. Trace:
I20260927 23:28:10.288584 3167 consensus_queue.cc:260] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.293169 2807 tablet_service.cc:1467] Processing CreateTablet for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc (DEFAULT_TABLE table=default.Table_eaa0db537a9f4c38b1c0db99654a6292 [id=9e14e6f54a2b42d18d036d528bed10f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.294147 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.298607 2938 tablet_service.cc:1467] Processing CreateTablet for tablet 8b891bab5fb343b0bb94641aea7662f0 (DEFAULT_TABLE table=default.Table_79ed7e54234b47fc8f068e605aa6121a [id=b1bafaccca474a2c9d5a92260eab27a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.316705 3558 rpcz_store.cc:276] 0927 23:28:08.354662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.356011 (+ 1349us) service_pool.cc:227] Handling call
0927 23:28:10.284001 (+1927990us) ts_tablet_manager.cc:1145] Deleting tablet 7096e434ac9f4290ad58ba4f7e7c2513
0927 23:28:10.284646 (+ 645us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.284830 (+ 184us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1927484}
W20260927 23:28:10.318078 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 7096e434ac9f4290ad58ba4f7e7c2513 because the tablet was not found. No further retry: Not found: Tablet not found: 7096e434ac9f4290ad58ba4f7e7c2513
I20260927 23:28:10.321810 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc (DEFAULT_TABLE table=default.Table_eaa0db537a9f4c38b1c0db99654a6292 [id=9e14e6f54a2b42d18d036d528bed10f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.322988 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.323400 2937 tablet_service.cc:1467] Processing CreateTablet for tablet fc60d0c7a3794607814393f033238052 (DEFAULT_TABLE table=default.Table_da4f2d8a1534460a8d060ea730c5e3e8 [id=8684e02a2c7f4b7b9ae8a200a36129c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.325235 2653 tablet_service.cc:1467] Processing CreateTablet for tablet e81df7c7f1a04eb9904dd2a1a5a596c4 (DEFAULT_TABLE table=default.Table_d0bc6f0546da41d1b7ce9d874ad0b416 [id=f0f6097616f742d2a6e32550bb46537c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.326308 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e81df7c7f1a04eb9904dd2a1a5a596c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.330860 2805 tablet_service.cc:1467] Processing CreateTablet for tablet e81df7c7f1a04eb9904dd2a1a5a596c4 (DEFAULT_TABLE table=default.Table_d0bc6f0546da41d1b7ce9d874ad0b416 [id=f0f6097616f742d2a6e32550bb46537c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.331971 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e81df7c7f1a04eb9904dd2a1a5a596c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.336707 3558 tablet_replica.cc:330] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:10.349135 3558 raft_consensus.cc:2229] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:10.351776 3070 tablet_bootstrap.cc:492] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:10.352257 3070 ts_tablet_manager.cc:1397] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.487s user 0.002s sys 0.019s
I20260927 23:28:10.351344 3693 tablet.cc:1722] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:10.353466 3167 ts_tablet_manager.cc:1428] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.105s user 0.004s sys 0.002s
I20260927 23:28:10.354471 3167 tablet_bootstrap.cc:492] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:10.371692 3558 raft_consensus.cc:2258] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:10.374435 3558 ts_tablet_manager.cc:1905] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:10.382212 2393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet ee5d073c8df84f0884dc0f0d88eeed9f
I20260927 23:28:10.385911 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet ee5d073c8df84f0884dc0f0d88eeed9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:10.392376 2935 tablet_service.cc:1467] Processing CreateTablet for tablet 355a04c4302441958d04cba33033076a (DEFAULT_TABLE table=default.Table_477638f0398348fdbb82d985e67e7c14 [id=67464814408f41fabc3895c474bb42f7]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:10.393383 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 119331us
I20260927 23:28:10.393487 3539 raft_consensus.cc:2781] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:10.393934 3539 raft_consensus.cc:482] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:10.394240 3539 raft_consensus.cc:3037] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:10.394567 3070 raft_consensus.cc:348] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.395252 3070 raft_consensus.cc:374] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:10.395535 3070 raft_consensus.cc:729] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
W20260927 23:28:10.400842 2378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc (table default.Table_eaa0db537a9f4c38b1c0db99654a6292 [id=9e14e6f54a2b42d18d036d528bed10f9], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc consensus state has no leader
I20260927 23:28:10.401813 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 04f670db808c4e289c655679da9d2d9d (table default.Table_9de2a295d1fc4fa692d084f283b81ba8 [id=bd8e999821b44f628e5ec3b1d7915773]) successfully deleted
I20260927 23:28:10.401885 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet ee5d073c8df84f0884dc0f0d88eeed9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:10.328258 2936 tablet_service.cc:1467] Processing CreateTablet for tablet 78e099028f7847d1b3de7d437be97de0 (DEFAULT_TABLE table=default.Table_29404653bfbb4e5d8192bcb604084df0 [id=baad95258e1a4358a58abc3fc1cc3737]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.406173 3167 tablet_bootstrap.cc:654] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:10.421976 2647 tablet_service.cc:1467] Processing CreateTablet for tablet 9af9b87abf1f4f78b3140a91f2525066 (DEFAULT_TABLE table=default.Table_5d215a67dc7943088d8ccb5a4691e8b6 [id=5e610846ba4f4cd7907236b723747067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.423097 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9af9b87abf1f4f78b3140a91f2525066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.420071 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 5dead3654e7a4b0da0e66593b33ade31 (DEFAULT_TABLE table=default.Table_4e3a8231cd1a426f82b9fecd2dc11400 [id=d5c8cc3b26ad49a8a2f90a05bafc86dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.425263 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dead3654e7a4b0da0e66593b33ade31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.425530 2387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_92a19ba603ef488c972786364cf49c70" } new_table_name: "default.Table_f7b5c90ef9a94341a88b0a40ea0f7e1e" modify_external_catalogs: true
I20260927 23:28:10.426970 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_11f58699a34140ff8ad27a712df72ab2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:10.428280 2388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 26007ba8846f4b08b337c3ce03782f4a
I20260927 23:28:10.431833 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_65cd17e1885c4ba6b11fb71b7c19ab4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:10.433298 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_65cd17e1885c4ba6b11fb71b7c19ab4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:10.433459 2652 tablet_service.cc:1514] Processing DeleteTablet for tablet 26007ba8846f4b08b337c3ce03782f4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:10.433525 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_11f58699a34140ff8ad27a712df72ab2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:10.435130 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_a64c5f39792f45aa84f17ca9bc9ab19b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:10.436488 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a64c5f39792f45aa84f17ca9bc9ab19b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:10.437539 2934 tablet_service.cc:1467] Processing CreateTablet for tablet c848ed8a221a4b799529ccc0d1830893 (DEFAULT_TABLE table=default.Table_9f0dec29a5d74ddaa48ef255c54b22cd [id=6f128a8acb5d47d69ead26501bd788bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.437740 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2a39980f0c46c68becefacad890ab7" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:10.440614 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "702ebdeb7c5a4b7d900dbb00236cb8ae" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:10.441947 2906 leader_election.cc:341] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:10.443486 3539 raft_consensus.cc:504] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.446919 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 84d37dba66b84d05abf675d3eaf39a30 (table default.Table_238b25adc2074249b812f76518fd76df [id=6671aa9b745f4863982b1d2dc75e5b63]). Expected version 1 got 0
I20260927 23:28:10.396111 3070 consensus_queue.cc:260] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.447938 2933 tablet_service.cc:1467] Processing CreateTablet for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc (DEFAULT_TABLE table=default.Table_eaa0db537a9f4c38b1c0db99654a6292 [id=9e14e6f54a2b42d18d036d528bed10f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.450389 3070 ts_tablet_manager.cc:1428] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.098s user 0.000s sys 0.005s
I20260927 23:28:10.450948 2932 tablet_service.cc:1514] Processing DeleteTablet for tablet ee5d073c8df84f0884dc0f0d88eeed9f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:10.451417 3070 tablet_bootstrap.cc:492] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:10.451611 3572 leader_election.cc:288] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:10.453042 3072 tablet_bootstrap.cc:492] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:10.453408 3072 ts_tablet_manager.cc:1397] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 1.015s user 0.020s sys 0.013s
I20260927 23:28:10.467804 2385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet bbeeccc19a504e7984b293127766daf0
I20260927 23:28:10.469123 2385 catalog_manager.cc:6565] ReplaceTablet: tablet bbeeccc19a504e7984b293127766daf0 deleted and replaced by tablet e268bcb4b0614f37a4d9d96f8f40d1cb
I20260927 23:28:10.469875 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet bbeeccc19a504e7984b293127766daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet e268bcb4b0614f37a4d9d96f8f40d1cb) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:10.474049 2384 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 35892219ae7140e58c96707d08190d4d has no consensus state
I20260927 23:28:10.477983 2802 tablet_service.cc:1467] Processing CreateTablet for tablet 5dead3654e7a4b0da0e66593b33ade31 (DEFAULT_TABLE table=default.Table_4e3a8231cd1a426f82b9fecd2dc11400 [id=d5c8cc3b26ad49a8a2f90a05bafc86dd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.479252 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dead3654e7a4b0da0e66593b33ade31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.481860 3539 leader_election.cc:288] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:10.475534 3070 tablet_bootstrap.cc:654] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:10.493824 2800 tablet_service.cc:1467] Processing CreateTablet for tablet 9af9b87abf1f4f78b3140a91f2525066 (DEFAULT_TABLE table=default.Table_5d215a67dc7943088d8ccb5a4691e8b6 [id=5e610846ba4f4cd7907236b723747067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.483225 3072 raft_consensus.cc:348] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.496579 3072 raft_consensus.cc:374] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:10.496857 3072 raft_consensus.cc:729] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:10.500626 2800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9af9b87abf1f4f78b3140a91f2525066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.506162 2801 tablet_service.cc:1514] Processing DeleteTablet for tablet bbeeccc19a504e7984b293127766daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet e268bcb4b0614f37a4d9d96f8f40d1cb) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:10.524204 2392 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_238b25adc2074249b812f76518fd76df [id=6671aa9b745f4863982b1d2dc75e5b63] alter complete (version 1)
I20260927 23:28:10.525695 3559 ts_tablet_manager.cc:1918] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:10.526158 3559 log.cc:1196] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/a80bff324902498fa5c04fe8cbc05f9d
W20260927 23:28:10.527287 2387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1da38499af164d7fb5c1a8d65167ad24 consensus state has no leader
I20260927 23:28:10.540124 2390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 3c6e596517324300be30df8506a18904
I20260927 23:28:10.542166 2390 catalog_manager.cc:6565] ReplaceTablet: tablet 3c6e596517324300be30df8506a18904 deleted and replaced by tablet df3a69b419ee4f37a2f6f9bffc7dd2dc
I20260927 23:28:10.547139 2805 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c6e596517324300be30df8506a18904 with delete_type TABLET_DATA_DELETED (replaced by tablet df3a69b419ee4f37a2f6f9bffc7dd2dc) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:10.519979 3072 consensus_queue.cc:260] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:10.555953 3693 tablet_replica.cc:1402] Time spent flushing the Log queue took a long time: real 0.203s user 0.000s sys 0.000s
I20260927 23:28:10.567570 3558 ts_tablet_manager.cc:1918] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:10.567963 3558 log.cc:1196] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/66f2842218fc42e5b1b83314e5f1a158
I20260927 23:28:10.526842 3559 ts_tablet_manager.cc:1939] T a80bff324902498fa5c04fe8cbc05f9d P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:10.568694 3558 ts_tablet_manager.cc:1939] T 66f2842218fc42e5b1b83314e5f1a158 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:10.568410 2647 tablet_service.cc:1467] Processing CreateTablet for tablet 73cfb41befe44e5189029abafe7e326d (DEFAULT_TABLE table=default.Table_b967cf863a6848919c6542cb6bbb1c70 [id=597f1d374b754602850d1dbc3bd4e913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.542851 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c6e596517324300be30df8506a18904 with delete_type TABLET_DATA_DELETED (replaced by tablet df3a69b419ee4f37a2f6f9bffc7dd2dc) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:10.511354 2795 tablet_service.cc:1514] Processing DeleteTablet for tablet 26007ba8846f4b08b337c3ce03782f4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:10.567800 2646 tablet_service.cc:1514] Processing DeleteTablet for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae with delete_type TABLET_DATA_DELETED (replaced by tablet 8fadaafd49b04b9483a6a50f2f7d5be7) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:10.570462 3167 tablet_bootstrap.cc:492] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:10.570842 3167 ts_tablet_manager.cc:1397] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.217s user 0.012s sys 0.003s
I20260927 23:28:10.570899 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73cfb41befe44e5189029abafe7e326d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.572978 3167 raft_consensus.cc:348] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.573614 3167 raft_consensus.cc:374] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:10.573841 3167 raft_consensus.cc:729] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:10.574515 3167 consensus_queue.cc:260] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.577154 3167 ts_tablet_manager.cc:1428] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260927 23:28:10.578890 2964 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.2.57.1:44363 (request call id 167) took 1027 ms. Trace:
I20260927 23:28:10.582180 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 7453cee17fa94dcda6f9ed4f48da26a5 (DEFAULT_TABLE table=default.Table_2342ac0b757f483bb8c19981aab80740 [id=227681dc1d7b4caeb5e077ea6ef69644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.584336 2932 tablet_service.cc:1467] Processing CreateTablet for tablet e81df7c7f1a04eb9904dd2a1a5a596c4 (DEFAULT_TABLE table=default.Table_d0bc6f0546da41d1b7ce9d874ad0b416 [id=f0f6097616f742d2a6e32550bb46537c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.589264 2795 tablet_service.cc:1467] Processing CreateTablet for tablet 73cfb41befe44e5189029abafe7e326d (DEFAULT_TABLE table=default.Table_b967cf863a6848919c6542cb6bbb1c70 [id=597f1d374b754602850d1dbc3bd4e913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.591801 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 155) took 1972 ms. Trace:
I20260927 23:28:10.592548 2761 leader_election.cc:302] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:10.592214 3559 rpcz_store.cc:276] 0927 23:28:08.619414 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.619558 (+ 144us) service_pool.cc:227] Handling call
0927 23:28:09.911839 (+1292281us) ts_tablet_manager.cc:1145] Deleting tablet a80bff324902498fa5c04fe8cbc05f9d
0927 23:28:09.911877 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.182892 (+271015us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.525454 (+342562us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.591716 (+ 66262us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5376,"tablet-delete.queue_time_us":1291804}
I20260927 23:28:10.593504 3635 raft_consensus.cc:2726] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:10.593621 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet a80bff324902498fa5c04fe8cbc05f9d (table default.Table_a5f98162932247a3ae2928ce03a691a4 [id=51372c7a0c1b4c9db9df0e9b0cbbe071]) successfully deleted
I20260927 23:28:10.596153 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 156) took 1975 ms. Trace:
I20260927 23:28:10.596519 3559 rpcz_store.cc:276] 0927 23:28:08.620426 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.620538 (+ 112us) service_pool.cc:227] Handling call
0927 23:28:10.595835 (+1975297us) ts_tablet_manager.cc:1145] Deleting tablet 3acdfe7faa1a4d5d8e340f09f1cae93e
0927 23:28:10.595882 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.596090 (+ 208us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1974774}
W20260927 23:28:10.600696 3694 log.cc:924] Time spent T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d: Append to log took a long time: real 0.076s user 0.000s sys 0.000s
I20260927 23:28:10.579314 2964 rpcz_store.cc:276] 0927 23:28:09.527795 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.612247 (+ 84452us) service_pool.cc:227] Handling call
0927 23:28:09.612779 (+ 532us) raft_consensus.cc:1457] Updating replica for 1 ops
0927 23:28:09.959229 (+346450us) raft_consensus.cc:1520] Early marking committed up to index 0
0927 23:28:09.969146 (+ 9917us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0927 23:28:10.178006 (+208860us) log.cc:842] Serialized 26 byte log entry
0927 23:28:10.346310 (+168304us) raft_consensus.cc:1631] Marking committed up to 1
0927 23:28:10.346958 (+ 648us) log.cc:842] Serialized 14 byte log entry
0927 23:28:10.347256 (+ 298us) raft_consensus.cc:1683] Filling consensus response to leader.
0927 23:28:10.347504 (+ 248us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0927 23:28:10.497396 (+149892us) raft_consensus.cc:1671] finished
0927 23:28:10.519696 (+ 22300us) raft_consensus.cc:1677] UpdateReplicas() finished
0927 23:28:10.555136 (+ 35440us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":16389,"prepare.queue_time_us":188,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":37545,"raft.run_cpu_time_us":437,"raft.run_wall_time_us":11116,"thread_start_us":346082,"threads_started":2,"wal-append.queue_time_us":167689}
I20260927 23:28:10.603216 3072 ts_tablet_manager.cc:1428] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.150s user 0.005s sys 0.003s
I20260927 23:28:10.611778 3559 tablet_replica.cc:330] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:10.612752 3559 raft_consensus.cc:2229] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:10.613626 3559 raft_consensus.cc:2258] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:10.615609 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 219656us
I20260927 23:28:10.623361 2944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0c12031a734ca0a6536404409c62ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.623967 3336 tablet_replica.cc:330] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:10.634853 3559 ts_tablet_manager.cc:1905] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:10.635756 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 7453cee17fa94dcda6f9ed4f48da26a5 (DEFAULT_TABLE table=default.Table_2342ac0b757f483bb8c19981aab80740 [id=227681dc1d7b4caeb5e077ea6ef69644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.641592 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7453cee17fa94dcda6f9ed4f48da26a5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:10.627256 2907 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.869s user 0.011s sys 0.004s
W20260927 23:28:10.643289 2963 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s user 0.003s sys 0.022s
W20260927 23:28:10.643569 2963 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.534s user 0.003s sys 0.022s
I20260927 23:28:10.645095 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_8fc13e9bc9e644779d259edabc03ecb6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:10.646713 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8fc13e9bc9e644779d259edabc03ecb6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:10.647194 2938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b891bab5fb343b0bb94641aea7662f0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:10.643018 2907 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.885s user 0.011s sys 0.004s
I20260927 23:28:10.649515 2392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_27f9b741e6184aa0bb0f95975292a4ae" } modify_external_catalogs: true
I20260927 23:28:10.652110 2389 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet d47c2f22abbe40e7ba8ea945ab96aeaf (table default.Table_41e19fe9cdcf42de870775c087f52478 [id=c8c0a7367a8342afb8ffc95627013a9c]). Expected version 1 got 0
I20260927 23:28:10.647634 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfbbe17f54d1418b95efc3943db0b6c1" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:10.654371 2968 raft_consensus.cc:2379] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:10.660805 2936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78e099028f7847d1b3de7d437be97de0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.667891 3336 raft_consensus.cc:2229] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:10.669229 3336 raft_consensus.cc:2258] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:10.661774 2392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_27f9b741e6184aa0bb0f95975292a4ae" } modify_external_catalogs: true
I20260927 23:28:10.674098 2937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc60d0c7a3794607814393f033238052. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.675777 2935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 355a04c4302441958d04cba33033076a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.677847 2934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c848ed8a221a4b799529ccc0d1830893. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.677716 2384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_646f1aa916434287ab21321062e1a5d1" } new_table_name: "default.Table_4f6bc587407d4e21a4a4a213a07b0ed8" modify_external_catalogs: true
W20260927 23:28:10.650738 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 892363us
I20260927 23:28:10.701385 2391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_9f0dec29a5d74ddaa48ef255c54b22cd" } modify_external_catalogs: true
I20260927 23:28:10.702061 2391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_9f0dec29a5d74ddaa48ef255c54b22cd" } modify_external_catalogs: true
I20260927 23:28:10.703464 2947 tablet_service.cc:1467] Processing CreateTablet for tablet 5dead3654e7a4b0da0e66593b33ade31 (DEFAULT_TABLE table=default.Table_4e3a8231cd1a426f82b9fecd2dc11400 [id=d5c8cc3b26ad49a8a2f90a05bafc86dd]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:10.704254 3703 log.cc:924] Time spent T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d: Append to log took a long time: real 0.057s user 0.000s sys 0.001s
I20260927 23:28:10.706465 2963 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.2.57.1:44363 (request call id 174) took 1169 ms. Trace:
I20260927 23:28:10.708204 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet 26007ba8846f4b08b337c3ce03782f4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:10.706774 2963 rpcz_store.cc:276] 0927 23:28:09.537250 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.628446 (+ 91196us) service_pool.cc:227] Handling call
0927 23:28:09.628935 (+ 489us) raft_consensus.cc:1457] Updating replica for 1 ops
0927 23:28:09.629215 (+ 280us) raft_consensus.cc:1520] Early marking committed up to index 0
0927 23:28:09.629239 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0927 23:28:09.636179 (+ 6940us) alter_schema_op.cc:126] START. Timestamp: 1790551689014890
0927 23:28:10.107815 (+471636us) spinlock_profiling.cc:243] Waited 93.5 ms on lock 0x7b2c00005568. stack: 00007f5a8686c033 00007f5a8686bf30 00007f5a863ed534 00000000004db2b4 00000000004db249 00007f5a8cae7e11 00007f5a8caf7442 00007f5a8d82ec11 00007f5a8d7f358a 00007f5a8d7f299e 00007f5a8e7cf2e9 00007f5a8cfa5684 00007f5a8cfa5612 00007f5a8cfa5541 00007f5a8cfa54bc 00007f5a8cfa4772
0927 23:28:10.108699 (+ 884us) log.cc:842] Serialized 173 byte log entry
0927 23:28:10.644278 (+535579us) raft_consensus.cc:1631] Marking committed up to 1
0927 23:28:10.644345 (+ 67us) raft_consensus.cc:1683] Filling consensus response to leader.
0927 23:28:10.644433 (+ 88us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0927 23:28:10.705767 (+ 61334us) raft_consensus.cc:1671] finished
0927 23:28:10.706259 (+ 492us) raft_consensus.cc:1677] UpdateReplicas() finished
0927 23:28:10.706394 (+ 135us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:09.885546 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0927 23:28:09.955116 (+ 69570us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0927 23:28:09.955148 (+ 32us) alter_schema_op.cc:65] Acquired schema lock
0927 23:28:09.955226 (+ 78us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
Metrics: {"spinlock_wait_cycles":215139072,"thread_start_us":534864,"threads_started":1,"wal-append.queue_time_us":534742,"child_traces":[["op",{"prepare.queue_time_us":195543,"prepare.run_cpu_time_us":638,"prepare.run_wall_time_us":69734,"thread_start_us":182880,"threads_started":1}]]}
I20260927 23:28:10.700498 2966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2a39980f0c46c68becefacad890ab7" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:10.710054 3336 ts_tablet_manager.cc:1905] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:10.711444 2966 raft_consensus.cc:3037] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:10.707930 2907 leader_election.cc:302] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:10.714216 2907 raft_consensus.cc:2670] T 3200d6bd5cb24ff5b1bbd59af9353dbe P de216a4a8e4b4e1b857d8647a3b05f6d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260927 23:28:10.714622 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet bbeeccc19a504e7984b293127766daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet e268bcb4b0614f37a4d9d96f8f40d1cb) from {username='slave'} at 127.0.0.1:44952
W20260927 23:28:10.717216 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 96682.4us
I20260927 23:28:10.720899 3539 consensus_queue.cc:1035] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:10.731074 3070 tablet_bootstrap.cc:492] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:10.731485 3070 ts_tablet_manager.cc:1397] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.280s user 0.040s sys 0.049s
I20260927 23:28:10.733099 2389 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_86d6643b68c849aea65e03a914f1d16f" } new_table_name: "default.Table_f580f5603b1b4c689ae82d0303ad8c4b" modify_external_catalogs: true
I20260927 23:28:10.734359 3070 raft_consensus.cc:348] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.733835 3559 ts_tablet_manager.cc:1918] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:10.735441 3559 log.cc:1196] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/04f670db808c4e289c655679da9d2d9d
I20260927 23:28:10.736217 3559 ts_tablet_manager.cc:1939] T 04f670db808c4e289c655679da9d2d9d P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:10.737046 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 158) took 2114 ms. Trace:
I20260927 23:28:10.737346 3559 rpcz_store.cc:276] 0927 23:28:08.623004 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.623101 (+ 97us) service_pool.cc:227] Handling call
0927 23:28:10.610360 (+1987259us) ts_tablet_manager.cc:1145] Deleting tablet 04f670db808c4e289c655679da9d2d9d
0927 23:28:10.611642 (+ 1282us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.665610 (+ 53968us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.722769 (+ 57159us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.736994 (+ 14225us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1986655}
W20260927 23:28:10.738168 2609 leader_election.cc:341] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:10.738546 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 04f670db808c4e289c655679da9d2d9d (table default.Table_9de2a295d1fc4fa692d084f283b81ba8 [id=bd8e999821b44f628e5ec3b1d7915773]) successfully deleted
I20260927 23:28:10.734957 3070 raft_consensus.cc:374] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:10.739143 3070 raft_consensus.cc:729] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:10.739902 3070 consensus_queue.cc:260] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.745309 3559 tablet_replica.cc:330] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:10.745594 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 9af9b87abf1f4f78b3140a91f2525066 (DEFAULT_TABLE table=default.Table_5d215a67dc7943088d8ccb5a4691e8b6 [id=5e610846ba4f4cd7907236b723747067]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.748876 3070 ts_tablet_manager.cc:1428] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260927 23:28:10.750224 3559 raft_consensus.cc:2229] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:10.751073 3559 raft_consensus.cc:2258] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:10.753140 3559 ts_tablet_manager.cc:1905] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:10.753783 3070 tablet_bootstrap.cc:492] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:10.755244 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_1458cf4426054ad583f77c955513541c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:10.756682 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1458cf4426054ad583f77c955513541c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:10.762074 3070 tablet_bootstrap.cc:654] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:10.764631 2943 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44952 (request call id 160) took 1148 ms. Trace:
I20260927 23:28:10.764945 2943 rpcz_store.cc:276] 0927 23:28:09.616569 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.626406 (+ 9837us) service_pool.cc:227] Handling call
0927 23:28:09.738920 (+112514us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:09.742015 (+ 3095us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:10.248688 (+506673us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.554873 (+306185us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:10.764563 (+209690us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:10.770035 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_a3208426010f46cf820de6a4dbba0099"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:10.771464 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a3208426010f46cf820de6a4dbba0099 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:10.774211 3693 tablet.cc:1722] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:10.774523 2950 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c6e596517324300be30df8506a18904 with delete_type TABLET_DATA_DELETED (replaced by tablet df3a69b419ee4f37a2f6f9bffc7dd2dc) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:10.776350 2950 tablet_service.cc:1467] Processing CreateTablet for tablet 73cfb41befe44e5189029abafe7e326d (DEFAULT_TABLE table=default.Table_b967cf863a6848919c6542cb6bbb1c70 [id=597f1d374b754602850d1dbc3bd4e913]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.786136 3559 ts_tablet_manager.cc:1918] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:10.786515 3559 log.cc:1196] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/7767a32c07454fe6a92154429a3b6899
I20260927 23:28:10.787177 3559 ts_tablet_manager.cc:1939] T 7767a32c07454fe6a92154429a3b6899 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:10.787922 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 159) took 2139 ms. Trace:
I20260927 23:28:10.788249 3559 rpcz_store.cc:276] 0927 23:28:08.648152 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.648266 (+ 114us) service_pool.cc:227] Handling call
0927 23:28:10.745138 (+2096872us) ts_tablet_manager.cc:1145] Deleting tablet 7767a32c07454fe6a92154429a3b6899
0927 23:28:10.745184 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.772218 (+ 27034us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.785899 (+ 13681us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.787872 (+ 1973us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2096233}
I20260927 23:28:10.793975 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 7767a32c07454fe6a92154429a3b6899 (table default.Table_6568f80896124508811607305bf18ff9 [id=adbf504950b64a08996f102f4ac2d662]) successfully deleted
I20260927 23:28:10.796352 3559 tablet_replica.cc:330] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:10.797143 3559 raft_consensus.cc:2229] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:10.797592 3559 raft_consensus.cc:2258] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:10.807605 2949 consensus_meta.cc:287] T 972763afd57847269fafadbdd2a5f3d6 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent flushing consensus metadata: real 0.845s user 0.011s sys 0.000s
W20260927 23:28:10.813395 2939 consensus_meta.cc:287] T 4d5b59f73974430c91ba19c2ec307ea0 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent flushing consensus metadata: real 0.710s user 0.010s sys 0.000s
I20260927 23:28:10.813617 3559 ts_tablet_manager.cc:1905] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:10.821698 2941 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44952 (request call id 165) took 1080 ms. Trace:
I20260927 23:28:10.822057 2941 rpcz_store.cc:276] 0927 23:28:09.743645 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.750304 (+ 6659us) service_pool.cc:227] Handling call
0927 23:28:09.822945 (+ 72641us) spinlock_profiling.cc:243] Waited 7.73 ms on lock 0x7b5800001898. stack: 00007f5a8686c033 00007f5a8686bf30 00007f5a863ed534 00000000004db2b4 00000000004db249 00007f5a866e2f11 00007f5a86866851 00007f5a86866702 00007f5a8f6fdb09 00007f5a8e75a730 00007f5a8e83cd21 00007f5a8e81a1e3 00007f5a8e7cb869 00007f5a8cebdc44 00007f5a8cebdbd2 00007f5a8cebdb01
0927 23:28:09.853503 (+ 30558us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b5800001898. stack: 00007f5a8686c033 00007f5a8686bf30 00007f5a863ed534 00000000004db2b4 00000000004db249 00007f5a866e2f11 00007f5a86866b7c 00007f5a8686673b 00007f5a8f6fdb09 00007f5a8e75a730 00007f5a8e83cd21 00007f5a8e81a1e3 00007f5a8e7cb869 00007f5a8cebdc44 00007f5a8cebdbd2 00007f5a8cebdb01
0927 23:28:09.969030 (+115527us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:09.985061 (+ 16031us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:10.405691 (+420630us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.580670 (+174979us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:10.821637 (+240967us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5224,"spinlock_wait_cycles":80462080}
W20260927 23:28:10.800963 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 99898.7us
I20260927 23:28:10.832165 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2a39980f0c46c68becefacad890ab7" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:10.832873 2907 leader_election.cc:341] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:10.833243 2907 leader_election.cc:302] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:10.834216 2907 leader_election.cc:302] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:10.835153 2907 leader_election.cc:341] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:10.835503 2907 leader_election.cc:302] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:10.839107 3072 tablet_bootstrap.cc:492] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:10.842238 2932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e81df7c7f1a04eb9904dd2a1a5a596c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.844738 3559 ts_tablet_manager.cc:1918] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:10.845108 3559 log.cc:1196] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/a4277045ec344ad7a48db9b14bea5e7b
I20260927 23:28:10.845775 3559 ts_tablet_manager.cc:1939] T a4277045ec344ad7a48db9b14bea5e7b P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:10.846446 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 160) took 2197 ms. Trace:
I20260927 23:28:10.846711 3559 rpcz_store.cc:276] 0927 23:28:08.649338 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.649435 (+ 97us) service_pool.cc:227] Handling call
0927 23:28:10.796165 (+2146730us) ts_tablet_manager.cc:1145] Deleting tablet a4277045ec344ad7a48db9b14bea5e7b
0927 23:28:10.796209 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.825859 (+ 29650us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.844516 (+ 18657us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.846399 (+ 1883us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2144865}
I20260927 23:28:10.847954 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7453cee17fa94dcda6f9ed4f48da26a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.847839 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet a4277045ec344ad7a48db9b14bea5e7b (table default.Table_a902e331f690448a872f6c104857f2ca [id=1acae7217ece44919a8d247f5cf155d1]) successfully deleted
I20260927 23:28:10.853008 2795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73cfb41befe44e5189029abafe7e326d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.853880 3559 tablet_replica.cc:330] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:10.854598 3559 raft_consensus.cc:2229] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:10.855077 3559 raft_consensus.cc:2258] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:10.860010 3559 ts_tablet_manager.cc:1905] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:10.865701 3167 tablet_bootstrap.cc:492] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:10.870814 3167 tablet_bootstrap.cc:654] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:10.874053 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 131) took 2520 ms. Trace:
I20260927 23:28:10.874383 3558 rpcz_store.cc:276] 0927 23:28:08.353553 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.362943 (+ 9390us) service_pool.cc:227] Handling call
0927 23:28:10.336038 (+1973095us) ts_tablet_manager.cc:1145] Deleting tablet 66f2842218fc42e5b1b83314e5f1a158
0927 23:28:10.336133 (+ 95us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.521343 (+185210us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.557726 (+ 36383us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.873992 (+316266us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":21538,"tablet-delete.queue_time_us":1947874}
I20260927 23:28:10.875481 3558 tablet_replica.cc:330] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:10.876272 3558 raft_consensus.cc:2229] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:10.876871 2933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.877213 3558 raft_consensus.cc:2258] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:10.879585 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 66f2842218fc42e5b1b83314e5f1a158 (table default.Table_33057639607344a0bcd2f4850a382324 [id=6b1d660c32a74492bb7b47fea2e1132d]) successfully deleted
I20260927 23:28:10.880820 3558 ts_tablet_manager.cc:1905] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:10.885169 3559 ts_tablet_manager.cc:1918] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:10.885514 3559 log.cc:1196] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/3200d6bd5cb24ff5b1bbd59af9353dbe
I20260927 23:28:10.886080 3559 ts_tablet_manager.cc:1939] T 3200d6bd5cb24ff5b1bbd59af9353dbe P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:10.886730 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 161) took 2026 ms. Trace:
I20260927 23:28:10.887007 3559 rpcz_store.cc:276] 0927 23:28:08.861345 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.864376 (+ 3031us) service_pool.cc:227] Handling call
0927 23:28:10.853737 (+1989361us) ts_tablet_manager.cc:1145] Deleting tablet 3200d6bd5cb24ff5b1bbd59af9353dbe
0927 23:28:10.853777 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.868927 (+ 15150us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.884973 (+ 16046us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.886685 (+ 1712us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1988767}
I20260927 23:28:10.888183 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 3200d6bd5cb24ff5b1bbd59af9353dbe (table default.Table_e57880e3ffa64d3b876813b8c2246002 [id=805a37d784f64efe9c46c985e5c853f1]) successfully deleted
I20260927 23:28:10.895113 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dead3654e7a4b0da0e66593b33ade31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.895273 3559 tablet_replica.cc:330] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:10.896663 3559 raft_consensus.cc:2229] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:10.898037 3559 pending_rounds.cc:62] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1: Trying to abort 1 pending ops.
I20260927 23:28:10.898773 3559 pending_rounds.cc:68] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099720964063232 op_type: NO_OP noop_request { }
I20260927 23:28:10.899647 3559 raft_consensus.cc:2866] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:28:10.900488 3559 raft_consensus.cc:2258] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:10.901185 2940 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44952 (request call id 167) took 1071 ms. Trace:
I20260927 23:28:10.901537 2940 rpcz_store.cc:276] 0927 23:28:09.830024 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.830199 (+ 175us) service_pool.cc:227] Handling call
0927 23:28:09.863081 (+ 32882us) spinlock_profiling.cc:243] Waited 2.17 ms on lock 0x7b5800001898. stack: 00007f5a8686c033 00007f5a8686bf30 00007f5a863ed534 00000000004db2b4 00000000004db249 00007f5a866e2f11 00007f5a86866851 00007f5a86866702 00007f5a8f6fdb09 00007f5a8e75a730 00007f5a8e83cd21 00007f5a8e81a1e3 00007f5a8e7cb869 00007f5a8cebdc44 00007f5a8cebdbd2 00007f5a8cebdb01
0927 23:28:09.992373 (+129292us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:09.992486 (+ 113us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:10.300943 (+308457us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.714966 (+414023us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:10.901124 (+186158us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4985600}
I20260927 23:28:10.903803 3559 ts_tablet_manager.cc:1905] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:10.904198 3680 raft_consensus.cc:2781] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:10.846485 3572 raft_consensus.cc:2726] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:10.911846 2950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73cfb41befe44e5189029abafe7e326d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.931982 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9af9b87abf1f4f78b3140a91f2525066. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:10.932331 3070 tablet_bootstrap.cc:492] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:10.932739 3070 ts_tablet_manager.cc:1397] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.179s user 0.024s sys 0.052s
I20260927 23:28:10.935664 3559 ts_tablet_manager.cc:1918] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:10.936415 3559 log.cc:1196] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/ac19099dd7bf4b669d504a654ee69482
I20260927 23:28:10.937093 3559 ts_tablet_manager.cc:1939] T ac19099dd7bf4b669d504a654ee69482 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:10.937783 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 162) took 2042 ms. Trace:
I20260927 23:28:10.938067 3559 rpcz_store.cc:276] 0927 23:28:08.900453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.908355 (+ 7902us) service_pool.cc:227] Handling call
0927 23:28:10.895128 (+1986773us) ts_tablet_manager.cc:1145] Deleting tablet ac19099dd7bf4b669d504a654ee69482
0927 23:28:10.895169 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.915792 (+ 20623us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.935465 (+ 19673us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.937732 (+ 2267us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1985976}
I20260927 23:28:10.939324 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet ac19099dd7bf4b669d504a654ee69482 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]) successfully deleted
I20260927 23:28:10.940440 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 166) took 1989 ms. Trace:
I20260927 23:28:10.940939 3559 rpcz_store.cc:276] 0927 23:28:08.950634 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.957360 (+ 6726us) service_pool.cc:227] Handling call
0927 23:28:10.940001 (+1982641us) ts_tablet_manager.cc:1145] Deleting tablet 2411d7c53a154fafad9edda9fa75bb36
0927 23:28:10.940037 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.940372 (+ 335us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1982042}
W20260927 23:28:10.942238 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 2411d7c53a154fafad9edda9fa75bb36 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2411d7c53a154fafad9edda9fa75bb36 already in progress: creating tablet
I20260927 23:28:10.943312 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 172) took 1930 ms. Trace:
I20260927 23:28:10.943688 3559 rpcz_store.cc:276] 0927 23:28:09.012789 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.013159 (+ 370us) service_pool.cc:227] Handling call
0927 23:28:10.942994 (+1929835us) ts_tablet_manager.cc:1145] Deleting tablet 5bf856a05dff4fb4943fe8e1d9b7f385
0927 23:28:10.943034 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.943264 (+ 230us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1929119}
W20260927 23:28:10.944823 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5bf856a05dff4fb4943fe8e1d9b7f385 already in progress: creating tablet
I20260927 23:28:10.951126 3559 tablet_replica.cc:330] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:10.952030 3559 raft_consensus.cc:2229] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:10.978785 2386 master_service.cc:946] ReplaceTablet: received request to replace tablet 972763afd57847269fafadbdd2a5f3d6 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:10.979125 3559 pending_rounds.cc:62] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1: Trying to abort 1 pending ops.
I20260927 23:28:10.979349 3559 pending_rounds.cc:68] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099720421855232 op_type: NO_OP noop_request { }
I20260927 23:28:10.979679 3559 raft_consensus.cc:2866] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20260927 23:28:10.979741 2384 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet bc9fb7e66b6b45c29688d9379e24d95d has no consensus state
I20260927 23:28:10.979992 3559 raft_consensus.cc:2258] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:10.981069 3070 raft_consensus.cc:348] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.981686 3070 raft_consensus.cc:374] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:10.981931 3070 raft_consensus.cc:729] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:10.983683 3070 consensus_queue.cc:260] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:10.985486 3070 ts_tablet_manager.cc:1428] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.052s user 0.005s sys 0.000s
I20260927 23:28:10.985904 2392 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet e268bcb4b0614f37a4d9d96f8f40d1cb
I20260927 23:28:10.986861 3559 ts_tablet_manager.cc:1905] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:10.987493 3683 log.cc:924] Time spent T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260927 23:28:10.974925 2939 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44952 (request call id 166) took 1207 ms. Trace:
I20260927 23:28:10.988387 2391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet c848ed8a221a4b799529ccc0d1830893
I20260927 23:28:10.988253 2939 rpcz_store.cc:276] 0927 23:28:09.766064 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.766196 (+ 132us) service_pool.cc:227] Handling call
0927 23:28:09.851779 (+ 85583us) spinlock_profiling.cc:243] Waited 81.5 ms on lock 0x7b5800001898. stack: 00007f5a8686c033 00007f5a8686bf30 00007f5a863ed534 00000000004db2b4 00000000004db249 00007f5a866e2f11 00007f5a86866851 00007f5a86866702 00007f5a8f6fdb09 00007f5a8e75a730 00007f5a8e83cd21 00007f5a8e81a1e3 00007f5a8e7cb869 00007f5a8cebdc44 00007f5a8cebdbd2 00007f5a8cebdb01
0927 23:28:09.967438 (+115659us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:09.967541 (+ 103us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:10.103211 (+135670us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.813809 (+710598us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:10.973404 (+159595us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":187546368}
I20260927 23:28:10.988921 3070 tablet_bootstrap.cc:492] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:10.989490 2943 tablet_service.cc:1467] Processing CreateTablet for tablet 7453cee17fa94dcda6f9ed4f48da26a5 (DEFAULT_TABLE table=default.Table_2342ac0b757f483bb8c19981aab80740 [id=227681dc1d7b4caeb5e077ea6ef69644]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:10.990706 2646 tablet_service.cc:1514] Processing DeleteTablet for tablet c848ed8a221a4b799529ccc0d1830893 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:10 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:10.993960 3680 raft_consensus.cc:686] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:10.995329 3680 consensus_queue.cc:237] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.001423 3070 tablet_bootstrap.cc:654] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:10.972389 2949 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44952 (request call id 164) took 1285 ms. Trace:
W20260927 23:28:11.004079 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 83082.1us
I20260927 23:28:11.017491 3696 raft_consensus.cc:482] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:11.017938 3696 raft_consensus.cc:504] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.019797 3696 leader_election.cc:288] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:11.024401 2761 leader_election.cc:302] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
W20260927 23:28:11.026574 2609 leader_election.cc:341] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:11.030011 3070 tablet_bootstrap.cc:492] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:11.030381 3070 ts_tablet_manager.cc:1397] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.042s user 0.009s sys 0.000s
I20260927 23:28:11.021932 2800 tablet_service.cc:1514] Processing DeleteTablet for tablet c848ed8a221a4b799529ccc0d1830893 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:10 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:11.032508 2821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b7eb04f8ff4726b4e9fe3d048ddafc" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:11.033679 2939 tablet_service.cc:1514] Processing DeleteTablet for tablet c848ed8a221a4b799529ccc0d1830893 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:10 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:11.031234 3635 raft_consensus.cc:3037] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.003882 2949 rpcz_store.cc:276] 0927 23:28:09.687409 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.689762 (+ 2353us) service_pool.cc:227] Handling call
0927 23:28:09.759148 (+ 69386us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:09.763367 (+ 4219us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:09.962490 (+199123us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:10.808028 (+845538us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:10.965908 (+157880us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:11.044919 2964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b7eb04f8ff4726b4e9fe3d048ddafc" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:11.045523 2964 raft_consensus.cc:2454] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
W20260927 23:28:11.047094 2609 leader_election.cc:341] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:11.048183 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7453cee17fa94dcda6f9ed4f48da26a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.054220 2966 raft_consensus.cc:2454] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:11.060796 3558 ts_tablet_manager.cc:1918] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:11.064920 3558 log.cc:1196] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/7767a32c07454fe6a92154429a3b6899
I20260927 23:28:11.064608 3070 raft_consensus.cc:348] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.065186 2391 master_service.cc:946] ReplaceTablet: received request to replace tablet 761a67c27c7a49a3903ed6f2bf5b96d3 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:11.065275 3070 raft_consensus.cc:374] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:11.065481 3070 raft_consensus.cc:729] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:11.063405 3635 raft_consensus.cc:2726] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:11.066864 2608 leader_election.cc:302] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:11.072928 2378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.60:41571: connect: Connection refused (error 111) [suppressed 5 similar messages]
I20260927 23:28:11.075971 3667 raft_consensus.cc:2726] T 0e2a39980f0c46c68becefacad890ab7 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:11.066074 3558 ts_tablet_manager.cc:1939] T 7767a32c07454fe6a92154429a3b6899 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:11.080322 3070 consensus_queue.cc:260] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.082486 3167 tablet_bootstrap.cc:492] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
W20260927 23:28:11.083397 2389 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ad8cdfaefc094c2394eb87f6bf33399e has no consensus state
I20260927 23:28:11.086055 3653 raft_consensus.cc:2781] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:11.086359 3653 raft_consensus.cc:482] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:11.086629 3653 raft_consensus.cc:3037] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.089953 3559 ts_tablet_manager.cc:1918] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:11.090301 3559 log.cc:1196] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/3f71f948891b4d7fa95c3a1fbd92ce1e
I20260927 23:28:11.093185 3559 ts_tablet_manager.cc:1939] T 3f71f948891b4d7fa95c3a1fbd92ce1e P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:11.094705 3653 raft_consensus.cc:504] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.095186 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 174) took 2064 ms. Trace:
I20260927 23:28:11.106288 3516 raft_consensus.cc:482] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:11.107375 3516 raft_consensus.cc:504] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.109577 3516 leader_election.cc:288] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:11.095479 3559 rpcz_store.cc:276] 0927 23:28:09.030693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.032494 (+ 1801us) service_pool.cc:227] Handling call
0927 23:28:10.950969 (+1918475us) ts_tablet_manager.cc:1145] Deleting tablet 3f71f948891b4d7fa95c3a1fbd92ce1e
0927 23:28:10.951014 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.978803 (+ 27789us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b58000728b0. stack: 00007fe1d7372033 00007fe1d7371f30 00007fe1d6ef3534 00000000004db2b4 00000000004db249 00007fe1de27e471 00007fe1de274f63 00007fe1de2d8aba 00007fe1de301375 00007fe1de71726c 00007fe1df324cfa 00007fe1df354e10 00007fe1df33be0d 00007fe1df33bdb9 00007fe1df33bd49 00007fe1df33bd11
0927 23:28:10.998930 (+ 20127us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.089742 (+ 90812us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.095133 (+ 5391us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":57049472,"tablet-delete.queue_time_us":1917701}
I20260927 23:28:11.115995 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 177) took 1993 ms. Trace:
I20260927 23:28:11.116565 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 3f71f948891b4d7fa95c3a1fbd92ce1e (table default.Table_047c5f89196f446e86279aae36cf7f99 [id=2aecf7c487414df1985aa5f284224515]) successfully deleted
I20260927 23:28:11.116276 3559 rpcz_store.cc:276] 0927 23:28:09.122368 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.123022 (+ 654us) service_pool.cc:227] Handling call
0927 23:28:11.115674 (+1992652us) ts_tablet_manager.cc:1145] Deleting tablet 8cd0972f635d4979abf5fd30e9b02b1d
0927 23:28:11.115714 (+ 40us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.115936 (+ 222us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1979668}
I20260927 23:28:11.117636 3559 tablet_replica.cc:330] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:11.118285 3559 raft_consensus.cc:2229] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
W20260927 23:28:11.118646 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 8cd0972f635d4979abf5fd30e9b02b1d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8cd0972f635d4979abf5fd30e9b02b1d already in progress: creating tablet
I20260927 23:28:11.126999 3070 ts_tablet_manager.cc:1428] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.096s user 0.006s sys 0.000s
I20260927 23:28:11.130177 3298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:28:11.132247 3559 pending_rounds.cc:62] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1: Trying to abort 1 pending ops.
I20260927 23:28:11.132526 3559 pending_rounds.cc:68] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099716800065536 op_type: NO_OP noop_request { }
I20260927 23:28:11.132928 3559 raft_consensus.cc:2866] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20260927 23:28:11.103847 2761 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 61297.3us
I20260927 23:28:11.133733 3559 raft_consensus.cc:2258] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:11.137458 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b7eb04f8ff4726b4e9fe3d048ddafc" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:11.138067 3559 ts_tablet_manager.cc:1905] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:11.138834 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0d8a82c5234880ab8396b1777e36c8" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:11.123262 3635 raft_consensus.cc:482] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:11.141902 3635 raft_consensus.cc:504] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.143359 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_86920ea4da9d4b3d85806280ca93e19e"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:11.144186 2609 leader_election.cc:341] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:11.144845 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_86920ea4da9d4b3d85806280ca93e19e in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:11.146083 2608 leader_election.cc:302] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:11.147482 3516 raft_consensus.cc:2781] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:11.147495 3298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:28:11.148185 3298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:28:11.149425 3516 raft_consensus.cc:686] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:11.084270 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 139) took 2672 ms. Trace:
I20260927 23:28:11.153579 3558 rpcz_store.cc:276] 0927 23:28:08.412177 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.412629 (+ 452us) service_pool.cc:227] Handling call
0927 23:28:10.875342 (+2462713us) ts_tablet_manager.cc:1145] Deleting tablet 7767a32c07454fe6a92154429a3b6899
0927 23:28:10.875380 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:10.908674 (+ 33294us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.060562 (+151888us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.084214 (+ 23652us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2462168}
I20260927 23:28:11.154805 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 7767a32c07454fe6a92154429a3b6899 (table default.Table_6568f80896124508811607305bf18ff9 [id=adbf504950b64a08996f102f4ac2d662]) successfully deleted
I20260927 23:28:11.155938 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 140) took 2743 ms. Trace:
I20260927 23:28:11.156265 3558 rpcz_store.cc:276] 0927 23:28:08.412494 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.434177 (+ 21683us) service_pool.cc:227] Handling call
0927 23:28:11.155542 (+2721365us) ts_tablet_manager.cc:1145] Deleting tablet 2411d7c53a154fafad9edda9fa75bb36
0927 23:28:11.155583 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.155890 (+ 307us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2693787}
I20260927 23:28:11.155792 2962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0d8a82c5234880ab8396b1777e36c8" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:11.142226 3667 raft_consensus.cc:482] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5ce671f028a34ea4b8f0520c2689c9a1)
I20260927 23:28:11.159822 3667 raft_consensus.cc:504] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } }
I20260927 23:28:11.161700 3667 leader_election.cc:288] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:11.173368 2424 debug-util.cc:398] Leaking SignalData structure 0x7b08000aafa0 after lost signal to thread 2359
W20260927 23:28:11.173763 2424 debug-util.cc:398] Leaking SignalData structure 0x7b0800098140 after lost signal to thread 2378
W20260927 23:28:11.174238 2424 debug-util.cc:398] Leaking SignalData structure 0x7b080004f280 after lost signal to thread 2427
I20260927 23:28:11.184175 2385 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_0ec64b963d5643a291b169781879ee55" } new_table_name: "default.Table_47fc9fc598ce4ca4bb4f94a523b39f74" modify_external_catalogs: true
W20260927 23:28:11.086004 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 250855us
I20260927 23:28:11.187522 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_4f8442f0e39b47a292bf291549e64212"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:11.190783 2389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_4f6bc587407d4e21a4a4a213a07b0ed8" } modify_external_catalogs: true
I20260927 23:28:11.191280 2389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_4f6bc587407d4e21a4a4a213a07b0ed8" } modify_external_catalogs: true
W20260927 23:28:11.191431 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 2411d7c53a154fafad9edda9fa75bb36 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2411d7c53a154fafad9edda9fa75bb36 already in progress: creating tablet
I20260927 23:28:11.192373 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 132) took 2838 ms. Trace:
I20260927 23:28:11.192677 3558 rpcz_store.cc:276] 0927 23:28:08.353833 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.468099 (+114266us) service_pool.cc:227] Handling call
0927 23:28:11.192109 (+2724010us) ts_tablet_manager.cc:1145] Deleting tablet cc1c0da6ea5c4a09a71555f37becae33
0927 23:28:11.192151 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.192323 (+ 172us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2723429}
W20260927 23:28:11.193681 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet cc1c0da6ea5c4a09a71555f37becae33 because the tablet was not found. No further retry: Not found: Tablet not found: cc1c0da6ea5c4a09a71555f37becae33
I20260927 23:28:11.132237 3167 ts_tablet_manager.cc:1397] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.267s user 0.029s sys 0.026s
I20260927 23:28:11.194571 3558 tablet_replica.cc:330] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:11.195268 3558 raft_consensus.cc:2229] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:11.195035 3693 tablet.cc:1722] T 84d37dba66b84d05abf675d3eaf39a30 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:11.195690 3558 raft_consensus.cc:2258] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:11.198125 2392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_6ff66acddae54d28b49ddde6a443bf7b" } modify_external_catalogs: true
I20260927 23:28:11.198642 2392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_6ff66acddae54d28b49ddde6a443bf7b" } modify_external_catalogs: true
W20260927 23:28:11.214534 2385 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 6d0d8a1f277242588f57e4df95df5d5a consensus state has no leader
W20260927 23:28:11.216264 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4f8442f0e39b47a292bf291549e64212 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:11.218586 2907 leader_election.cc:302] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:11.221083 3072 tablet_bootstrap.cc:654] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:11.223615 3558 ts_tablet_manager.cc:1905] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:11.229957 3713 raft_consensus.cc:2726] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260927 23:28:11.252367 2609 leader_election.cc:341] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:11.252825 2608 leader_election.cc:341] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:11.252602 3516 consensus_queue.cc:237] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.253497 2608 leader_election.cc:302] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:11.256546 3696 raft_consensus.cc:2726] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:11.257179 3653 leader_election.cc:288] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:11.257225 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 84226.8us
I20260927 23:28:11.256699 3070 tablet_bootstrap.cc:492] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:11.260970 2963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b7eb04f8ff4726b4e9fe3d048ddafc" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:11.261670 2963 raft_consensus.cc:3037] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.263077 3070 tablet_bootstrap.cc:654] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:11.227767 2907 leader_election.cc:341] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:11.265882 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 98912.6us
I20260927 23:28:11.279986 3721 tablet.cc:1722] T d47c2f22abbe40e7ba8ea945ab96aeaf P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:11.283074 3559 ts_tablet_manager.cc:1918] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:11.283447 3559 log.cc:1196] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/cfbbe17f54d1418b95efc3943db0b6c1
I20260927 23:28:11.288713 2388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_decc7c294d064d7daea036b101d04d55" } new_table_name: "default.Table_4fa6aec871b84978973cb579e195404b" modify_external_catalogs: true
I20260927 23:28:11.299626 3167 raft_consensus.cc:348] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.284152 3559 ts_tablet_manager.cc:1939] T cfbbe17f54d1418b95efc3943db0b6c1 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
W20260927 23:28:11.302960 2763 leader_election.cc:341] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 3200d6bd5cb24ff5b1bbd59af9353dbe
I20260927 23:28:11.303486 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 178) took 2163 ms. Trace:
I20260927 23:28:11.303782 3559 rpcz_store.cc:276] 0927 23:28:09.140089 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.141414 (+ 1325us) service_pool.cc:227] Handling call
0927 23:28:11.117498 (+1976084us) ts_tablet_manager.cc:1145] Deleting tablet cfbbe17f54d1418b95efc3943db0b6c1
0927 23:28:11.117542 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.252747 (+135205us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.282854 (+ 30107us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.303431 (+ 20577us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1975474}
I20260927 23:28:11.304919 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet cfbbe17f54d1418b95efc3943db0b6c1 (table default.Table_96276d84ee11493d89908e4ff16dc7fe [id=f21b53d8e90a48bc9cb7f6268c946cc1]) successfully deleted
I20260927 23:28:11.305570 3635 leader_election.cc:288] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:11.309517 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_1ad5ebc0110d45a280c889a7b7670506"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:11.310976 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1ad5ebc0110d45a280c889a7b7670506 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:11.312913 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 175) took 2208 ms. Trace:
W20260927 23:28:11.313428 2906 leader_election.cc:341] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Not found: Tablet not found: 2d837daee7a04b338813519acf53dfae
I20260927 23:28:11.313228 3559 rpcz_store.cc:276] 0927 23:28:09.112521 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.121724 (+ 9203us) service_pool.cc:227] Handling call
0927 23:28:11.311402 (+2189678us) ts_tablet_manager.cc:1145] Deleting tablet f58349edf0db40f98951e116fc2582b0
0927 23:28:11.311440 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.311687 (+ 247us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2139143}
W20260927 23:28:11.314314 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet f58349edf0db40f98951e116fc2582b0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f58349edf0db40f98951e116fc2582b0 already in progress: creating tablet
I20260927 23:28:11.316412 3167 raft_consensus.cc:374] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:11.316708 3167 raft_consensus.cc:729] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:11.316569 2384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_cd24927b2fec4eee916e402106212f33"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:11.317319 3167 consensus_queue.cc:260] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:11.318003 2384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_cd24927b2fec4eee916e402106212f33 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:11.319622 2388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1c87484b8b9742faa683f13d37308e58 consensus state has no leader
I20260927 23:28:11.320576 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 181) took 1741 ms. Trace:
I20260927 23:28:11.322624 3167 ts_tablet_manager.cc:1428] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.128s user 0.005s sys 0.001s
I20260927 23:28:11.325717 3167 tablet_bootstrap.cc:492] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:11.320866 3559 rpcz_store.cc:276] 0927 23:28:09.579585 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.579717 (+ 132us) service_pool.cc:227] Handling call
0927 23:28:11.320176 (+1740459us) ts_tablet_manager.cc:1145] Deleting tablet fd0115095a82489a8aeb941efcc8a979
0927 23:28:11.320262 (+ 86us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.320519 (+ 257us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1739844}
W20260927 23:28:11.326519 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet fd0115095a82489a8aeb941efcc8a979 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fd0115095a82489a8aeb941efcc8a979 already in progress: creating tablet
I20260927 23:28:11.328621 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 182) took 1737 ms. Trace:
W20260927 23:28:11.329048 2761 leader_election.cc:341] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: 3200d6bd5cb24ff5b1bbd59af9353dbe
I20260927 23:28:11.329414 2761 leader_election.cc:302] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:11.330265 3635 raft_consensus.cc:2726] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:11.330617 3167 tablet_bootstrap.cc:654] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:11.281823 2907 leader_election.cc:302] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:11.331600 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 105227us
I20260927 23:28:11.332052 3516 raft_consensus.cc:482] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:28:11.332722 2907 leader_election.cc:341] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:11.332566 3516 raft_consensus.cc:504] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.333092 2907 leader_election.cc:302] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:11.334149 3572 raft_consensus.cc:2726] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:11.334705 3516 leader_election.cc:288] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:11.330183 3559 rpcz_store.cc:276] 0927 23:28:09.591023 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.591165 (+ 142us) service_pool.cc:227] Handling call
0927 23:28:11.328282 (+1737117us) ts_tablet_manager.cc:1145] Deleting tablet 8b891bab5fb343b0bb94641aea7662f0
0927 23:28:11.328341 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.328569 (+ 228us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1736557}
W20260927 23:28:11.335912 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 8b891bab5fb343b0bb94641aea7662f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b891bab5fb343b0bb94641aea7662f0 already in progress: creating tablet
I20260927 23:28:11.336213 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1912a82ff46f4f9aaa24b7621cc3995d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:11.337702 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 184) took 1661 ms. Trace:
W20260927 23:28:11.338241 2609 leader_election.cc:341] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:11.340631 2937 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44952 (request call id 177) took 1085 ms. Trace:
I20260927 23:28:11.340981 2937 rpcz_store.cc:276] 0927 23:28:10.255268 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.311636 (+ 56368us) service_pool.cc:227] Handling call
0927 23:28:10.623661 (+312025us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:10.623778 (+ 117us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:11.137706 (+513928us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.326615 (+188909us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:11.340560 (+ 13945us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:11.350935 2962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1912a82ff46f4f9aaa24b7621cc3995d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:11.351552 2962 raft_consensus.cc:2454] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:11.354228 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "702ebdeb7c5a4b7d900dbb00236cb8ae" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:11.354789 2669 raft_consensus.cc:3037] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.361956 3070 tablet_bootstrap.cc:492] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:11.362452 3070 ts_tablet_manager.cc:1397] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.106s user 0.005s sys 0.005s
I20260927 23:28:11.365155 3070 raft_consensus.cc:348] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.368642 3558 ts_tablet_manager.cc:1918] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:11.372118 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_de478761aba74f9bab295c750c11178d"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:11.373579 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de478761aba74f9bab295c750c11178d in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:11.380466 3558 log.cc:1196] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/a80bff324902498fa5c04fe8cbc05f9d
I20260927 23:28:11.384869 3558 ts_tablet_manager.cc:1939] T a80bff324902498fa5c04fe8cbc05f9d P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:11.337996 3559 rpcz_store.cc:276] 0927 23:28:09.676623 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.677363 (+ 740us) service_pool.cc:227] Handling call
0927 23:28:11.337362 (+1659999us) ts_tablet_manager.cc:1145] Deleting tablet a1890852164d492db3540a97847888fd
0927 23:28:11.337400 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.337651 (+ 251us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1656496}
W20260927 23:28:11.387106 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet a1890852164d492db3540a97847888fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1890852164d492db3540a97847888fd already in progress: creating tablet
W20260927 23:28:11.388909 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet ee5d073c8df84f0884dc0f0d88eeed9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee5d073c8df84f0884dc0f0d88eeed9f already in progress: creating tablet
W20260927 23:28:11.390425 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 26007ba8846f4b08b337c3ce03782f4a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26007ba8846f4b08b337c3ce03782f4a already in progress: creating tablet
I20260927 23:28:11.392271 3559 tablet_replica.cc:330] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:11.393025 3559 raft_consensus.cc:2229] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:11.393535 3559 raft_consensus.cc:2258] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:11.395303 3559 ts_tablet_manager.cc:1905] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:11.399757 2963 raft_consensus.cc:2454] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:11.371744 2669 raft_consensus.cc:2454] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:11.400218 3713 raft_consensus.cc:2726] T 92b7eb04f8ff4726b4e9fe3d048ddafc P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:11.367641 2608 leader_election.cc:302] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:11.400911 3516 raft_consensus.cc:2781] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:11.401223 3516 raft_consensus.cc:482] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:11.401497 3516 raft_consensus.cc:3037] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.410643 2608 leader_election.cc:302] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:11.416138 3653 raft_consensus.cc:2781] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:11.416594 3653 raft_consensus.cc:686] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:11.417291 3653 consensus_queue.cc:237] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.386911 3167 tablet_bootstrap.cc:492] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:11.402421 3635 raft_consensus.cc:482] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:11.424427 3696 raft_consensus.cc:482] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:11.424505 3635 raft_consensus.cc:504] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.424821 3696 raft_consensus.cc:504] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.425623 3070 raft_consensus.cc:374] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:11.425917 3070 raft_consensus.cc:729] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:11.426355 3635 leader_election.cc:288] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:11.426545 3696 leader_election.cc:288] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:11.427079 3167 ts_tablet_manager.cc:1397] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.102s user 0.008s sys 0.001s
I20260927 23:28:11.428953 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf856a05dff4fb4943fe8e1d9b7f385" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:11.429371 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "702ebdeb7c5a4b7d900dbb00236cb8ae" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:11.429201 3167 raft_consensus.cc:348] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.429903 3167 raft_consensus.cc:374] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:11.430050 2971 raft_consensus.cc:2379] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate de216a4a8e4b4e1b857d8647a3b05f6d in this term.
I20260927 23:28:11.430130 3167 raft_consensus.cc:729] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
W20260927 23:28:11.430518 2609 leader_election.cc:341] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:11.428506 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "702ebdeb7c5a4b7d900dbb00236cb8ae" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:11.431555 2669 raft_consensus.cc:2379] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate de216a4a8e4b4e1b857d8647a3b05f6d in this term.
I20260927 23:28:11.430732 3167 consensus_queue.cc:260] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.432940 2763 leader_election.cc:302] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
W20260927 23:28:11.433706 2379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 6d0d8a1f277242588f57e4df95df5d5a (table default.Table_47fc9fc598ce4ca4bb4f94a523b39f74 [id=7b1bf5063eef401eb9de9b6f7aa40e4b], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 6d0d8a1f277242588f57e4df95df5d5a consensus state has no leader
I20260927 23:28:11.434100 3635 raft_consensus.cc:3037] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:11.432247 2907 consensus_peers.cc:487] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4929fc9e396e48a189c94586d58aa55f. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:11.438376 2907 leader_election.cc:341] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 2d837daee7a04b338813519acf53dfae
I20260927 23:28:11.441951 3167 ts_tablet_manager.cc:1428] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s
I20260927 23:28:11.443140 3635 raft_consensus.cc:2726] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:11.426503 3070 consensus_queue.cc:260] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.447108 3336 ts_tablet_manager.cc:1918] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:11.447484 3336 log.cc:1196] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/ac19099dd7bf4b669d504a654ee69482
I20260927 23:28:11.448048 3336 ts_tablet_manager.cc:1939] T ac19099dd7bf4b669d504a654ee69482 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:11.451207 3167 tablet_bootstrap.cc:492] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:11.454273 3070 ts_tablet_manager.cc:1428] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.092s user 0.005s sys 0.000s
I20260927 23:28:11.455580 3070 tablet_bootstrap.cc:492] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:11.438789 2907 leader_election.cc:302] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:11.463403 3621 raft_consensus.cc:2726] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:11.465806 2907 leader_election.cc:302] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:11.466480 3621 raft_consensus.cc:2781] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:11.467412 3621 raft_consensus.cc:686] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:11.468122 3621 consensus_queue.cc:237] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.479307 2963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf856a05dff4fb4943fe8e1d9b7f385" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:11.479961 2963 raft_consensus.cc:2454] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:11.415143 3516 raft_consensus.cc:504] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.481992 3516 leader_election.cc:288] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:11.483541 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1912a82ff46f4f9aaa24b7621cc3995d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:11.485659 2609 leader_election.cc:341] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:11.485906 3167 tablet_bootstrap.cc:654] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:11.465507 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 130) took 3092 ms. Trace:
I20260927 23:28:11.487074 3558 rpcz_store.cc:276] 0927 23:28:08.353221 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.468764 (+115543us) service_pool.cc:227] Handling call
0927 23:28:11.194438 (+2725674us) ts_tablet_manager.cc:1145] Deleting tablet a80bff324902498fa5c04fe8cbc05f9d
0927 23:28:11.194472 (+ 34us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.339504 (+145032us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.368438 (+ 28934us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.445564 (+ 77126us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2724861}
I20260927 23:28:11.488487 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet a80bff324902498fa5c04fe8cbc05f9d (table default.Table_a5f98162932247a3ae2928ce03a691a4 [id=51372c7a0c1b4c9db9df0e9b0cbbe071]) successfully deleted
I20260927 23:28:11.489487 3558 tablet_replica.cc:330] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:11.490362 3635 raft_consensus.cc:482] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:11.491834 3635 raft_consensus.cc:504] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.494642 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e596517324300be30df8506a18904" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:11.495955 2763 leader_election.cc:341] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: Tablet not RUNNING: STOPPED
I20260927 23:28:11.496502 3072 tablet_bootstrap.cc:492] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:11.496896 3072 ts_tablet_manager.cc:1397] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.658s user 0.013s sys 0.010s
I20260927 23:28:11.497275 3635 leader_election.cc:288] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:11.498490 3558 raft_consensus.cc:2229] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:11.498948 3558 raft_consensus.cc:2258] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:11.499505 2963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e596517324300be30df8506a18904" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:11.503190 3070 tablet_bootstrap.cc:654] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:11.504678 3558 ts_tablet_manager.cc:1905] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:11.505940 2963 raft_consensus.cc:2379] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate de216a4a8e4b4e1b857d8647a3b05f6d in this term.
I20260927 23:28:11.507310 2388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_70bf888024144eb0b5afd48ee484fb17" } new_table_name: "default.Table_7113798bfe2c4627ac0f51de8a85e8d2" modify_external_catalogs: true
I20260927 23:28:11.502337 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 109) took 6222 ms. Trace:
I20260927 23:28:11.504870 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1912a82ff46f4f9aaa24b7621cc3995d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:11.512652 2971 raft_consensus.cc:3037] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.510528 3167 tablet_bootstrap.cc:492] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:11.512709 2761 leader_election.cc:302] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:11.517565 3635 raft_consensus.cc:3037] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.517215 2384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_60becdf63fbc49b9a338f109290d00c8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:11.518616 2384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_60becdf63fbc49b9a338f109290d00c8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:11.523975 3635 raft_consensus.cc:2726] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:11.524667 3167 ts_tablet_manager.cc:1397] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.074s user 0.008s sys 0.002s
I20260927 23:28:11.528132 3336 rpcz_store.cc:276] 0927 23:28:05.279848 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:05.279944 (+ 96us) service_pool.cc:227] Handling call
0927 23:28:10.268650 (+4988706us) ts_tablet_manager.cc:1145] Deleting tablet ac19099dd7bf4b669d504a654ee69482
0927 23:28:10.623041 (+354391us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.074645 (+451604us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.446776 (+372131us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.502286 (+ 55510us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":40964,"tablet-delete.queue_time_us":4988184}
I20260927 23:28:11.529724 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet ac19099dd7bf4b669d504a654ee69482 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]) successfully deleted
I20260927 23:28:11.531080 2608 leader_election.cc:302] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:11.528497 3167 raft_consensus.cc:348] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.536923 2971 raft_consensus.cc:2454] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:11.537925 3516 raft_consensus.cc:2781] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:11.538276 3516 raft_consensus.cc:482] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:11.538616 3516 raft_consensus.cc:3037] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.533840 3167 raft_consensus.cc:374] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:11.540650 3167 raft_consensus.cc:729] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:11.543239 3167 consensus_queue.cc:260] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.545291 3516 raft_consensus.cc:504] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.549391 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf856a05dff4fb4943fe8e1d9b7f385" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:11.550514 2609 leader_election.cc:341] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:11.551810 3653 raft_consensus.cc:482] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:11.552315 3653 raft_consensus.cc:504] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.555765 3653 leader_election.cc:288] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:11.556754 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0115095a82489a8aeb941efcc8a979" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:11.557615 3070 tablet_bootstrap.cc:492] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
W20260927 23:28:11.557965 2609 leader_election.cc:341] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:11.557978 3070 ts_tablet_manager.cc:1397] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.103s user 0.003s sys 0.006s
I20260927 23:28:11.558974 3336 tablet_replica.cc:330] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:11.559803 3336 raft_consensus.cc:2229] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:11.560320 3336 raft_consensus.cc:2258] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:11.560158 3070 raft_consensus.cc:348] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.560781 3070 raft_consensus.cc:374] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:11.561009 3070 raft_consensus.cc:729] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:11.562482 3336 ts_tablet_manager.cc:1905] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:11.564145 3167 ts_tablet_manager.cc:1428] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.039s user 0.005s sys 0.000s
I20260927 23:28:11.565111 3559 ts_tablet_manager.cc:1918] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:11.565475 3559 log.cc:1196] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/3c6e596517324300be30df8506a18904
I20260927 23:28:11.566051 3559 ts_tablet_manager.cc:1939] T 3c6e596517324300be30df8506a18904 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:11.567790 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 208) took 1025 ms. Trace:
I20260927 23:28:11.568583 3070 consensus_queue.cc:260] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.570487 3070 ts_tablet_manager.cc:1428] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.012s user 0.003s sys 0.000s
I20260927 23:28:11.571756 2608 leader_election.cc:302] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:11.572487 3696 raft_consensus.cc:2781] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:11.575098 3167 tablet_bootstrap.cc:492] T 92b7eb04f8ff4726b4e9fe3d048ddafc P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:11.579706 3167 tablet_bootstrap.cc:654] T 92b7eb04f8ff4726b4e9fe3d048ddafc P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:11.582049 3516 leader_election.cc:288] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:11.594542 2390 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_485140265fb4433280c5e7ae5316f122 [id=687de3cd507b4b5e82ba6df82a290757] alter complete (version 1)
I20260927 23:28:11.597780 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 52aa3ab7bd8f465ba3c6499814b26b38 (DEFAULT_TABLE table=default.Table_a64c5f39792f45aa84f17ca9bc9ab19b [id=a1793a44569940da852251cb520d5ce6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.598742 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52aa3ab7bd8f465ba3c6499814b26b38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.580848 3072 raft_consensus.cc:348] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.593457 2963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0115095a82489a8aeb941efcc8a979" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:11.603787 3072 raft_consensus.cc:374] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:11.604072 3072 raft_consensus.cc:729] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:11.606995 2647 tablet_service.cc:1467] Processing CreateTablet for tablet d49bbfa59bf14de587ee8d1230bcbf19 (DEFAULT_TABLE table=default.Table_f19cc1c4a83548278bc4b767ee445951 [id=a69ffd4b45a145dead445ccfe5d4ddee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.607915 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d49bbfa59bf14de587ee8d1230bcbf19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.598721 3070 tablet_bootstrap.cc:492] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:11.568058 3559 rpcz_store.cc:276] 0927 23:28:10.542557 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.542698 (+ 141us) service_pool.cc:227] Handling call
0927 23:28:11.392140 (+849442us) ts_tablet_manager.cc:1145] Deleting tablet 3c6e596517324300be30df8506a18904
0927 23:28:11.392175 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.516243 (+124068us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.551840 (+ 35597us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.567736 (+ 15896us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":822563}
I20260927 23:28:11.619390 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 3c6e596517324300be30df8506a18904 (table default.Table_585d1f792a1b42e7ae86d5ecb5edaff2 [id=ff09406f8316449ead4cffa47a2095b8]) successfully deleted
I20260927 23:28:11.625111 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 52aa3ab7bd8f465ba3c6499814b26b38 (DEFAULT_TABLE table=default.Table_a64c5f39792f45aa84f17ca9bc9ab19b [id=a1793a44569940da852251cb520d5ce6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.626331 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52aa3ab7bd8f465ba3c6499814b26b38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.627173 3070 tablet_bootstrap.cc:654] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:11.604898 3072 consensus_queue.cc:260] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.631839 3696 raft_consensus.cc:686] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:11.634436 3559 tablet_replica.cc:330] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:11.635138 3559 raft_consensus.cc:2229] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:11.635581 3559 raft_consensus.cc:2258] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:11.632802 3696 consensus_queue.cc:237] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.637141 3559 ts_tablet_manager.cc:1905] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:11.640847 2393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_9088c5760dc44415bf9a9106a40f5ab4" } new_table_name: "default.Table_48ee3ed6e26d4e41aa46aaf16ff3a09d" modify_external_catalogs: true
I20260927 23:28:11.641309 3072 ts_tablet_manager.cc:1428] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.144s user 0.006s sys 0.000s
I20260927 23:28:11.629062 2963 raft_consensus.cc:2454] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:11.645844 2943 tablet_service.cc:1467] Processing CreateTablet for tablet d49bbfa59bf14de587ee8d1230bcbf19 (DEFAULT_TABLE table=default.Table_f19cc1c4a83548278bc4b767ee445951 [id=a69ffd4b45a145dead445ccfe5d4ddee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.651473 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bf856a05dff4fb4943fe8e1d9b7f385" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:11.652016 2971 raft_consensus.cc:3037] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.657685 2608 leader_election.cc:302] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:11.658697 3749 raft_consensus.cc:2781] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:11.659039 3749 raft_consensus.cc:482] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:11.659320 3749 raft_consensus.cc:3037] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.654264 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d49bbfa59bf14de587ee8d1230bcbf19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.671069 2647 tablet_service.cc:1467] Processing CreateTablet for tablet 0ad117dfb8664938a9c8c7d0b200045e (DEFAULT_TABLE table=default.Table_73f1344829a046f29f209adc7eceaf42 [id=16e24fbc3d434613aeead047b9d624ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.672034 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ad117dfb8664938a9c8c7d0b200045e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.671285 3559 ts_tablet_manager.cc:1918] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:11.677565 3559 log.cc:1196] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/702ebdeb7c5a4b7d900dbb00236cb8ae
I20260927 23:28:11.678174 3559 ts_tablet_manager.cc:1939] T 702ebdeb7c5a4b7d900dbb00236cb8ae P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:11.679852 3559 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 209) took 1125 ms. Trace:
I20260927 23:28:11.683629 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 1 got 0
I20260927 23:28:11.684422 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:11.680130 3559 rpcz_store.cc:276] 0927 23:28:10.554760 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.567616 (+ 12856us) service_pool.cc:227] Handling call
0927 23:28:11.620962 (+1053346us) ts_tablet_manager.cc:1145] Deleting tablet 702ebdeb7c5a4b7d900dbb00236cb8ae
0927 23:28:11.634233 (+ 13271us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.653469 (+ 19236us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.671086 (+ 17617us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.679798 (+ 8712us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1050874}
I20260927 23:28:11.685770 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 702ebdeb7c5a4b7d900dbb00236cb8ae (table default.Table_f1571426f866470d94cfe827bd30ba79 [id=0bcc62622acc4fb7a60d137f1b75265d]) successfully deleted
W20260927 23:28:11.693303 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet c848ed8a221a4b799529ccc0d1830893 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c848ed8a221a4b799529ccc0d1830893 already in progress: creating tablet
I20260927 23:28:11.697875 3749 raft_consensus.cc:504] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:11.699525 2388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1625e523e1ca40b6b8324f302bb70d31 consensus state has no leader
I20260927 23:28:11.707202 3070 tablet_bootstrap.cc:492] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:11.707613 3070 ts_tablet_manager.cc:1397] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.109s user 0.019s sys 0.005s
I20260927 23:28:11.709702 3070 raft_consensus.cc:348] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.710309 3070 raft_consensus.cc:374] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:11.710561 3070 raft_consensus.cc:729] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:11.714287 2647 tablet_service.cc:1467] Processing CreateTablet for tablet 10fe6b0a4c614b77ae95b200b6a12843 (DEFAULT_TABLE table=default.Table_df694a00f8a8437cb1d2d2d01a107afc [id=b54a76b761cc4db6b9926451b0fa1db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.715225 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fe6b0a4c614b77ae95b200b6a12843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.715682 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 7a5274b56b814c029e02c16243a3e663 (DEFAULT_TABLE table=default.Table_6ff66acddae54d28b49ddde6a443bf7b [id=10c9dfbac3be4358be4dd66a584e9a68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.716655 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a5274b56b814c029e02c16243a3e663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.720423 3749 leader_election.cc:288] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:11.728000 3070 consensus_queue.cc:260] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.730648 3070 ts_tablet_manager.cc:1428] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.023s user 0.004s sys 0.000s
I20260927 23:28:11.731455 3070 tablet_bootstrap.cc:492] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:11.739300 3070 tablet_bootstrap.cc:654] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:11.739682 2947 tablet_service.cc:1467] Processing CreateTablet for tablet 10fe6b0a4c614b77ae95b200b6a12843 (DEFAULT_TABLE table=default.Table_df694a00f8a8437cb1d2d2d01a107afc [id=b54a76b761cc4db6b9926451b0fa1db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.741925 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0115095a82489a8aeb941efcc8a979" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:11.742604 2968 raft_consensus.cc:3037] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.746448 2932 tablet_service.cc:1467] Processing CreateTablet for tablet 0ad117dfb8664938a9c8c7d0b200045e (DEFAULT_TABLE table=default.Table_73f1344829a046f29f209adc7eceaf42 [id=16e24fbc3d434613aeead047b9d624ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.747779 2932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ad117dfb8664938a9c8c7d0b200045e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.749145 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 7a5274b56b814c029e02c16243a3e663 (DEFAULT_TABLE table=default.Table_6ff66acddae54d28b49ddde6a443bf7b [id=10c9dfbac3be4358be4dd66a584e9a68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.750381 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a5274b56b814c029e02c16243a3e663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.751497 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fe6b0a4c614b77ae95b200b6a12843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.790669 2968 raft_consensus.cc:2454] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:11.792347 2608 leader_election.cc:302] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:11.793282 3696 raft_consensus.cc:2781] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:11.793838 3696 raft_consensus.cc:686] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:11.794680 3696 consensus_queue.cc:237] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.796638 3070 tablet_bootstrap.cc:492] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:11.797067 3070 ts_tablet_manager.cc:1397] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.066s user 0.033s sys 0.024s
I20260927 23:28:11.839131 3572 raft_consensus.cc:482] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:11.839627 3572 raft_consensus.cc:504] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.841524 3572 leader_election.cc:288] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:11.843294 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0d8a82c5234880ab8396b1777e36c8" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:11.843989 2669 raft_consensus.cc:2454] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:11.845290 2907 leader_election.cc:302] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:11.846626 3572 raft_consensus.cc:2781] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:11.847028 3572 raft_consensus.cc:482] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:11.847406 3572 raft_consensus.cc:3037] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.854722 2647 tablet_service.cc:1467] Processing CreateTablet for tablet a1a14a6135dc466881c1182450d7d4f0 (DEFAULT_TABLE table=default.Table_ccc7b44c45f24b7c9b58a89e4b99f139 [id=1ac693cc0c31400293cbc935d9367ace]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.855854 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a14a6135dc466881c1182450d7d4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.861243 2946 tablet_service.cc:1467] Processing CreateTablet for tablet a1a14a6135dc466881c1182450d7d4f0 (DEFAULT_TABLE table=default.Table_ccc7b44c45f24b7c9b58a89e4b99f139 [id=1ac693cc0c31400293cbc935d9367ace]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.862437 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a14a6135dc466881c1182450d7d4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.897401 2971 raft_consensus.cc:2454] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:11.898636 2608 leader_election.cc:302] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:11.899981 3749 raft_consensus.cc:2781] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:11.900425 3749 raft_consensus.cc:686] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:11.901192 3749 consensus_queue.cc:237] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.911585 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_c79cb0e575124bc29051aa5905a314e9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:11.913421 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_c79cb0e575124bc29051aa5905a314e9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:11.936096 3072 tablet_bootstrap.cc:492] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:11.949419 3572 raft_consensus.cc:504] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.953912 2391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 761a67c27c7a49a3903ed6f2bf5b96d3
I20260927 23:28:11.955276 2391 catalog_manager.cc:6565] ReplaceTablet: tablet 761a67c27c7a49a3903ed6f2bf5b96d3 deleted and replaced by tablet f102a35faef94cfd880f5b41d3e610e7
I20260927 23:28:11.957931 2651 tablet_service.cc:1514] Processing DeleteTablet for tablet 761a67c27c7a49a3903ed6f2bf5b96d3 with delete_type TABLET_DATA_DELETED (replaced by tablet f102a35faef94cfd880f5b41d3e610e7) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:11.959933 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 761a67c27c7a49a3903ed6f2bf5b96d3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 761a67c27c7a49a3903ed6f2bf5b96d3 already in progress: creating tablet
I20260927 23:28:11.960028 2933 tablet_service.cc:1514] Processing DeleteTablet for tablet 761a67c27c7a49a3903ed6f2bf5b96d3 with delete_type TABLET_DATA_DELETED (replaced by tablet f102a35faef94cfd880f5b41d3e610e7) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:11.966195 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0d8a82c5234880ab8396b1777e36c8" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:11.966815 2669 raft_consensus.cc:3037] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.977615 2822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0d8a82c5234880ab8396b1777e36c8" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:11.978698 2795 tablet_service.cc:1467] Processing CreateTablet for tablet 52aa3ab7bd8f465ba3c6499814b26b38 (DEFAULT_TABLE table=default.Table_a64c5f39792f45aa84f17ca9bc9ab19b [id=a1793a44569940da852251cb520d5ce6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.981663 2800 tablet_service.cc:1467] Processing CreateTablet for tablet 0ad117dfb8664938a9c8c7d0b200045e (DEFAULT_TABLE table=default.Table_73f1344829a046f29f209adc7eceaf42 [id=16e24fbc3d434613aeead047b9d624ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.982771 2828 raft_consensus.cc:3037] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:11.983556 2800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ad117dfb8664938a9c8c7d0b200045e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.983935 2669 raft_consensus.cc:2454] T 2d0d8a82c5234880ab8396b1777e36c8 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:11.985538 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 7a5274b56b814c029e02c16243a3e663 (DEFAULT_TABLE table=default.Table_6ff66acddae54d28b49ddde6a443bf7b [id=10c9dfbac3be4358be4dd66a584e9a68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.987056 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a5274b56b814c029e02c16243a3e663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:11.884390 3070 raft_consensus.cc:348] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.988655 3070 raft_consensus.cc:374] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:11.988937 3070 raft_consensus.cc:729] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:11.587659 3558 ts_tablet_manager.cc:1918] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:11.989413 2802 tablet_service.cc:1467] Processing CreateTablet for tablet 10fe6b0a4c614b77ae95b200b6a12843 (DEFAULT_TABLE table=default.Table_df694a00f8a8437cb1d2d2d01a107afc [id=b54a76b761cc4db6b9926451b0fa1db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.989717 3070 consensus_queue.cc:260] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:11.990160 3558 log.cc:1196] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/3f71f948891b4d7fa95c3a1fbd92ce1e
I20260927 23:28:11.991181 3558 ts_tablet_manager.cc:1939] T 3f71f948891b4d7fa95c3a1fbd92ce1e P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:11.991433 2804 tablet_service.cc:1467] Processing CreateTablet for tablet a1a14a6135dc466881c1182450d7d4f0 (DEFAULT_TABLE table=default.Table_ccc7b44c45f24b7c9b58a89e4b99f139 [id=1ac693cc0c31400293cbc935d9367ace]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:11.991844 3070 ts_tablet_manager.cc:1428] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.195s user 0.005s sys 0.000s
I20260927 23:28:11.993367 2805 tablet_service.cc:1467] Processing CreateTablet for tablet d49bbfa59bf14de587ee8d1230bcbf19 (DEFAULT_TABLE table=default.Table_f19cc1c4a83548278bc4b767ee445951 [id=a69ffd4b45a145dead445ccfe5d4ddee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.001847 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd0115095a82489a8aeb941efcc8a979" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:12.002178 2906 leader_election.cc:341] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:12.001854 3635 raft_consensus.cc:482] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:12.003667 3749 raft_consensus.cc:482] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:12.004124 3749 raft_consensus.cc:504] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.003739 3635 raft_consensus.cc:504] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.006091 3749 leader_election.cc:288] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:12.007642 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 56f87f2226db4958a46f70e7bc1db899 (DEFAULT_TABLE table=default.Table_4c3e33fd8fa1454f89c1a524de0bcc39 [id=d2661975a7464fa683fe2dd9b13924f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.007982 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da38499af164d7fb5c1a8d65167ad24" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:12.008802 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56f87f2226db4958a46f70e7bc1db899. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:12.009389 2608 leader_election.cc:341] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:12.010272 2609 leader_election.cc:341] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:12.013715 2907 leader_election.cc:302] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:12.015518 3621 raft_consensus.cc:2781] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:12.015944 3621 raft_consensus.cc:686] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:12.016793 3621 consensus_queue.cc:237] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.027945 3572 leader_election.cc:288] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:12.032092 3072 tablet_bootstrap.cc:654] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:12.060777 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fe6b0a4c614b77ae95b200b6a12843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.069578 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0d8a82c5234880ab8396b1777e36c8" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:12.071061 2795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52aa3ab7bd8f465ba3c6499814b26b38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.073915 3635 leader_election.cc:288] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:12.074882 2906 leader_election.cc:341] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:12.075011 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1912a82ff46f4f9aaa24b7621cc3995d" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:12.075879 2971 raft_consensus.cc:2379] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
I20260927 23:28:12.076426 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1912a82ff46f4f9aaa24b7621cc3995d" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:12.060799 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da38499af164d7fb5c1a8d65167ad24" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:12.078011 2609 leader_election.cc:341] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:12.078419 2609 leader_election.cc:302] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:12.078352 2763 leader_election.cc:302] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:12.079483 3749 raft_consensus.cc:2726] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:12.079771 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d49bbfa59bf14de587ee8d1230bcbf19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.081839 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1a14a6135dc466881c1182450d7d4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.082405 3072 tablet_bootstrap.cc:492] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:12.082839 3072 ts_tablet_manager.cc:1397] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.192s user 0.010s sys 0.000s
I20260927 23:28:12.085345 2650 tablet_service.cc:1467] Processing CreateTablet for tablet e2d52ef453b847b386154da01537fbca (DEFAULT_TABLE table=default.Table_e4b17b19ddc8470a94bb58e6ac8e6750 [id=988acf1b83064b21b9a74ae0a714b1ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.086493 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2d52ef453b847b386154da01537fbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.091913 2947 tablet_service.cc:1467] Processing CreateTablet for tablet 56f87f2226db4958a46f70e7bc1db899 (DEFAULT_TABLE table=default.Table_4c3e33fd8fa1454f89c1a524de0bcc39 [id=d2661975a7464fa683fe2dd9b13924f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.093156 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56f87f2226db4958a46f70e7bc1db899. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:12.097767 2609 consensus_peers.cc:487] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:12.099891 2796 tablet_service.cc:1467] Processing CreateTablet for tablet 56f87f2226db4958a46f70e7bc1db899 (DEFAULT_TABLE table=default.Table_4c3e33fd8fa1454f89c1a524de0bcc39 [id=d2661975a7464fa683fe2dd9b13924f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.101172 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56f87f2226db4958a46f70e7bc1db899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.103623 2798 tablet_service.cc:1514] Processing DeleteTablet for tablet 761a67c27c7a49a3903ed6f2bf5b96d3 with delete_type TABLET_DATA_DELETED (replaced by tablet f102a35faef94cfd880f5b41d3e610e7) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:12.094069 3072 raft_consensus.cc:348] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.104363 3072 raft_consensus.cc:374] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:12.105525 3072 raft_consensus.cc:729] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:12.105320 2794 tablet_service.cc:1467] Processing CreateTablet for tablet e2d52ef453b847b386154da01537fbca (DEFAULT_TABLE table=default.Table_e4b17b19ddc8470a94bb58e6ac8e6750 [id=988acf1b83064b21b9a74ae0a714b1ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.106638 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2d52ef453b847b386154da01537fbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.112650 3072 consensus_queue.cc:260] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.119510 2389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet bc9fb7e66b6b45c29688d9379e24d95d
I20260927 23:28:12.125665 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet bc9fb7e66b6b45c29688d9379e24d95d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:11 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:12.126214 3167 tablet_bootstrap.cc:492] T 92b7eb04f8ff4726b4e9fe3d048ddafc P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:12.126744 3167 ts_tablet_manager.cc:1397] T 92b7eb04f8ff4726b4e9fe3d048ddafc P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.552s user 0.014s sys 0.005s
I20260927 23:28:12.127957 3749 raft_consensus.cc:482] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:12.128441 3749 raft_consensus.cc:504] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:12.129097 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet bc9fb7e66b6b45c29688d9379e24d95d because the tablet was not found. No further retry: Not found: Tablet not found: bc9fb7e66b6b45c29688d9379e24d95d
I20260927 23:28:12.130417 3749 leader_election.cc:288] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:12.131845 2392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 7a5274b56b814c029e02c16243a3e663
I20260927 23:28:12.133955 2932 tablet_service.cc:1514] Processing DeleteTablet for tablet bc9fb7e66b6b45c29688d9379e24d95d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:11 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:12.135963 2932 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a5274b56b814c029e02c16243a3e663 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:11 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:12.136889 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a5274b56b814c029e02c16243a3e663 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:11 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:12.138675 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 7a5274b56b814c029e02c16243a3e663 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a5274b56b814c029e02c16243a3e663 already in progress: creating tablet
I20260927 23:28:12.141768 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2411d7c53a154fafad9edda9fa75bb36" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:12.142746 3572 consensus_queue.cc:1035] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:12.143373 2828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2411d7c53a154fafad9edda9fa75bb36" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:12.144605 2608 leader_election.cc:341] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:12.145888 2609 leader_election.cc:341] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:12.146258 2609 leader_election.cc:302] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:12.147382 3749 raft_consensus.cc:2726] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:12.147570 2386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 972763afd57847269fafadbdd2a5f3d6
I20260927 23:28:12.148976 2386 catalog_manager.cc:6565] ReplaceTablet: tablet 972763afd57847269fafadbdd2a5f3d6 deleted and replaced by tablet 82ff8173836e4cf2b43bf7e816a26f0a
I20260927 23:28:12.149348 2386 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:51972 (request call id 250) took 1369 ms. Trace:
I20260927 23:28:12.149609 2386 rpcz_store.cc:276] 0927 23:28:10.779634 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.978672 (+199038us) service_pool.cc:227] Handling call
0927 23:28:11.041890 (+ 63218us) spinlock_profiling.cc:243] Waited 61.6 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b5b81 00007f8bb3c10d37 00007f8bb3c10cc2 00007f8bb3c10bf1 00007f8bb3c10b6c 00007f8bb3c0fe22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:12.149279 (+1107389us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:11.918318 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:11.918664 (+ 346us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:11.918686 (+ 22us) write_op.cc:435] Acquired schema lock
0927 23:28:11.918699 (+ 13us) tablet.cc:660] Decoding operations
0927 23:28:11.918989 (+ 290us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:11.919026 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:11.919047 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:11.919246 (+ 199us) tablet.cc:699] Row locks acquired
0927 23:28:11.919261 (+ 15us) write_op.cc:260] PREPARE: finished
0927 23:28:11.919393 (+ 132us) write_op.cc:270] Start()
0927 23:28:11.919481 (+ 88us) write_op.cc:276] Timestamp: P: 1790551691916235 usec, L: 0
0927 23:28:11.919499 (+ 18us) op_driver.cc:331] REPLICATION: starting
0927 23:28:11.919963 (+ 464us) log.cc:842] Serialized 619 byte log entry
0927 23:28:12.117170 (+197207us) op_driver.cc:444] REPLICATION: finished
0927 23:28:12.139941 (+ 22771us) write_op.cc:301] APPLY: starting
0927 23:28:12.139997 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:12.140107 (+ 110us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:12.140190 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:12.145322 (+ 5132us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:12.145662 (+ 340us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:12.145708 (+ 46us) write_op.cc:312] APPLY: finished
0927 23:28:12.145948 (+ 240us) log.cc:842] Serialized 29 byte log entry
0927 23:28:12.146775 (+ 827us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:12.146856 (+ 81us) write_op.cc:454] Released schema lock
0927 23:28:12.146892 (+ 36us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":141650688,"child_traces":[["op",{"apply.queue_time_us":22646,"num_ops":2,"prepare.queue_time_us":253849,"prepare.run_cpu_time_us":1820,"prepare.run_wall_time_us":1819,"replication_time_us":197614,"threads_started":1}]]}
I20260927 23:28:12.152381 3635 raft_consensus.cc:3037] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:12.152817 3072 ts_tablet_manager.cc:1428] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.070s user 0.004s sys 0.003s
I20260927 23:28:12.154212 3072 tablet_bootstrap.cc:492] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:12.156088 2932 tablet_service.cc:1514] Processing DeleteTablet for tablet 972763afd57847269fafadbdd2a5f3d6 with delete_type TABLET_DATA_DELETED (replaced by tablet 82ff8173836e4cf2b43bf7e816a26f0a) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:12.157830 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet 972763afd57847269fafadbdd2a5f3d6 with delete_type TABLET_DATA_DELETED (replaced by tablet 82ff8173836e4cf2b43bf7e816a26f0a) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:12.160742 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 972763afd57847269fafadbdd2a5f3d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 972763afd57847269fafadbdd2a5f3d6 already in progress: creating tablet
I20260927 23:28:12.023736 2384 catalog_manager.cc:5495] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:12.181255 3072 tablet_bootstrap.cc:654] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:12.181464 2932 tablet_service.cc:1467] Processing CreateTablet for tablet e2d52ef453b847b386154da01537fbca (DEFAULT_TABLE table=default.Table_e4b17b19ddc8470a94bb58e6ac8e6750 [id=988acf1b83064b21b9a74ae0a714b1ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.182662 2932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2d52ef453b847b386154da01537fbca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.186118 2799 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a5274b56b814c029e02c16243a3e663 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:11 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:12.187059 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet 972763afd57847269fafadbdd2a5f3d6 with delete_type TABLET_DATA_DELETED (replaced by tablet 82ff8173836e4cf2b43bf7e816a26f0a) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:12.187747 2798 tablet_service.cc:1514] Processing DeleteTablet for tablet bc9fb7e66b6b45c29688d9379e24d95d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:11 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:12.190753 3167 raft_consensus.cc:348] T 92b7eb04f8ff4726b4e9fe3d048ddafc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.191288 3749 consensus_queue.cc:1035] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:12.195217 3070 tablet_bootstrap.cc:492] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:12.232110 2798 tablet_service.cc:1467] Processing CreateTablet for tablet d9b629c1bbff4ea7b215b5b2c07d9f87 (DEFAULT_TABLE table=default.Table_f7dab7db47874af6b9466b544d9b8465 [id=79d1c5d616e4492baee2d600d4f77b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.233518 2798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9b629c1bbff4ea7b215b5b2c07d9f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.191669 3167 raft_consensus.cc:374] T 92b7eb04f8ff4726b4e9fe3d048ddafc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:12.240648 3167 raft_consensus.cc:729] T 92b7eb04f8ff4726b4e9fe3d048ddafc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:12.241509 3167 consensus_queue.cc:260] T 92b7eb04f8ff4726b4e9fe3d048ddafc P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.208413 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_de44500abb18471d93e90c264b5491d5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:12.250749 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_de44500abb18471d93e90c264b5491d5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:12.209813 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_fdb186a310414586b0e653d8ef8dfeb0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:12.255096 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_fdb186a310414586b0e653d8ef8dfeb0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:12.212776 2386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_63f9354acd7d4ca3ad92ab1edf9d7174" } new_table_name: "default.Table_0df33121d2eb4445ac8b8c6a9ee44d24" modify_external_catalogs: true
I20260927 23:28:12.264389 3635 raft_consensus.cc:2726] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:12.272799 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 147) took 3725 ms. Trace:
I20260927 23:28:12.273222 3558 rpcz_store.cc:276] 0927 23:28:08.547366 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.556387 (+ 9021us) service_pool.cc:227] Handling call
0927 23:28:11.489286 (+2932899us) ts_tablet_manager.cc:1145] Deleting tablet 3f71f948891b4d7fa95c3a1fbd92ce1e
0927 23:28:11.489330 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.546263 (+ 56933us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:11.587455 (+ 41192us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:12.272685 (+685230us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2932357}
I20260927 23:28:12.274533 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 3f71f948891b4d7fa95c3a1fbd92ce1e (table default.Table_047c5f89196f446e86279aae36cf7f99 [id=2aecf7c487414df1985aa5f284224515]) successfully deleted
I20260927 23:28:12.275995 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 148) took 3728 ms. Trace:
I20260927 23:28:12.276495 3558 rpcz_store.cc:276] 0927 23:28:08.547625 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.557023 (+ 9398us) service_pool.cc:227] Handling call
0927 23:28:12.275512 (+3718489us) ts_tablet_manager.cc:1145] Deleting tablet f58349edf0db40f98951e116fc2582b0
0927 23:28:12.275589 (+ 77us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:12.275922 (+ 333us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3717959}
W20260927 23:28:12.277675 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet f58349edf0db40f98951e116fc2582b0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f58349edf0db40f98951e116fc2582b0 already in progress: creating tablet
I20260927 23:28:12.278712 2647 tablet_service.cc:1467] Processing CreateTablet for tablet d9b629c1bbff4ea7b215b5b2c07d9f87 (DEFAULT_TABLE table=default.Table_f7dab7db47874af6b9466b544d9b8465 [id=79d1c5d616e4492baee2d600d4f77b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.279844 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9b629c1bbff4ea7b215b5b2c07d9f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.287865 3572 consensus_queue.cc:1035] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:12.315827 2943 tablet_service.cc:1467] Processing CreateTablet for tablet d9b629c1bbff4ea7b215b5b2c07d9f87 (DEFAULT_TABLE table=default.Table_f7dab7db47874af6b9466b544d9b8465 [id=79d1c5d616e4492baee2d600d4f77b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.270164 2384 catalog_manager.cc:5495] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:12.321293 2384 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:12.327077 3072 tablet_bootstrap.cc:492] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:12.335050 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9b629c1bbff4ea7b215b5b2c07d9f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.336385 2932 tablet_service.cc:1467] Processing CreateTablet for tablet fa65be0c3c4347a48a53fa2a3f6e7544 (DEFAULT_TABLE table=default.Table_59054c732de9495093fd5a97281b4cf5 [id=fe1f8e25be8b40beace6ec0818f026ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.337479 2932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa65be0c3c4347a48a53fa2a3f6e7544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.335340 3072 ts_tablet_manager.cc:1397] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.181s user 0.009s sys 0.011s
I20260927 23:28:12.364146 2807 tablet_service.cc:1467] Processing CreateTablet for tablet fa65be0c3c4347a48a53fa2a3f6e7544 (DEFAULT_TABLE table=default.Table_59054c732de9495093fd5a97281b4cf5 [id=fe1f8e25be8b40beace6ec0818f026ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.365345 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa65be0c3c4347a48a53fa2a3f6e7544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.384661 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 145) took 3865 ms. Trace:
I20260927 23:28:12.385066 3558 rpcz_store.cc:276] 0927 23:28:08.546740 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.560745 (+ 14005us) service_pool.cc:227] Handling call
0927 23:28:12.339322 (+3778577us) ts_tablet_manager.cc:1145] Deleting tablet 5bf856a05dff4fb4943fe8e1d9b7f385
0927 23:28:12.362524 (+ 23202us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:12.384582 (+ 22058us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3757654}
W20260927 23:28:12.386369 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5bf856a05dff4fb4943fe8e1d9b7f385 already in progress: creating tablet
I20260927 23:28:12.387688 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 150) took 3657 ms. Trace:
I20260927 23:28:12.388000 3558 rpcz_store.cc:276] 0927 23:28:08.730129 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.733034 (+ 2905us) service_pool.cc:227] Handling call
0927 23:28:12.387249 (+3654215us) ts_tablet_manager.cc:1145] Deleting tablet 8cd0972f635d4979abf5fd30e9b02b1d
0927 23:28:12.387318 (+ 69us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:12.387631 (+ 313us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3653352}
W20260927 23:28:12.389460 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 8cd0972f635d4979abf5fd30e9b02b1d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8cd0972f635d4979abf5fd30e9b02b1d already in progress: creating tablet
I20260927 23:28:12.340586 2384 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:12.393587 2384 catalog_manager.cc:5495] T 0e2a39980f0c46c68becefacad890ab7 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:12.395553 2384 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1da38499af164d7fb5c1a8d65167ad24 (table default.Table_f7b5c90ef9a94341a88b0a40ea0f7e1e [id=b12aae65f933476ab57974d4b328a266]). Expected version 3 got 0
I20260927 23:28:12.411151 3558 tablet_replica.cc:330] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:12.427671 2946 tablet_service.cc:1467] Processing CreateTablet for tablet df3a69b419ee4f37a2f6f9bffc7dd2dc (DEFAULT_TABLE table=default.Table_585d1f792a1b42e7ae86d5ecb5edaff2 [id=ff09406f8316449ead4cffa47a2095b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.428993 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df3a69b419ee4f37a2f6f9bffc7dd2dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.434826 2388 master_service.cc:946] ReplaceTablet: received request to replace tablet 10fe6b0a4c614b77ae95b200b6a12843 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:12.329551 3336 ts_tablet_manager.cc:1918] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:12.330993 3572 raft_consensus.cc:482] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:12.439591 2392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_da4f2d8a1534460a8d060ea730c5e3e8" } modify_external_catalogs: true
I20260927 23:28:12.440181 2392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_da4f2d8a1534460a8d060ea730c5e3e8" } modify_external_catalogs: true
I20260927 23:28:12.440021 3572 raft_consensus.cc:504] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.442121 3572 leader_election.cc:288] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:12.445228 2906 leader_election.cc:341] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Not found: Tablet not found: 3f71f948891b4d7fa95c3a1fbd92ce1e
I20260927 23:28:12.445724 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_b41d59628db149279b4f71e292a05f98"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:12.447198 2907 leader_election.cc:341] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 3f71f948891b4d7fa95c3a1fbd92ce1e
W20260927 23:28:12.447309 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b41d59628db149279b4f71e292a05f98 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:12.447599 2907 leader_election.cc:302] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:12.448910 3572 raft_consensus.cc:2726] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:12.449740 3336 log.cc:1196] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/74fc2f03baae4f70b38b8079deb8a854
I20260927 23:28:12.450138 2801 tablet_service.cc:1467] Processing CreateTablet for tablet bc9fb7e66b6b45c29688d9379e24d95d (DEFAULT_TABLE table=default.Table_4f6bc587407d4e21a4a4a213a07b0ed8 [id=c54e791a750941d79f74e5a943b8a2b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.450624 3336 ts_tablet_manager.cc:1939] T 74fc2f03baae4f70b38b8079deb8a854 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:12.451685 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc9fb7e66b6b45c29688d9379e24d95d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.351998 3072 raft_consensus.cc:348] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.453397 2389 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_5d215a67dc7943088d8ccb5a4691e8b6" } modify_external_catalogs: true
I20260927 23:28:12.453926 2389 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_5d215a67dc7943088d8ccb5a4691e8b6" } modify_external_catalogs: true
I20260927 23:28:12.455770 2805 tablet_service.cc:1467] Processing CreateTablet for tablet df3a69b419ee4f37a2f6f9bffc7dd2dc (DEFAULT_TABLE table=default.Table_585d1f792a1b42e7ae86d5ecb5edaff2 [id=ff09406f8316449ead4cffa47a2095b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.456571 3072 raft_consensus.cc:374] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:12.456934 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df3a69b419ee4f37a2f6f9bffc7dd2dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.456929 3072 raft_consensus.cc:729] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:12.460865 3167 ts_tablet_manager.cc:1428] T 92b7eb04f8ff4726b4e9fe3d048ddafc P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.334s user 0.014s sys 0.003s
I20260927 23:28:12.459528 3072 consensus_queue.cc:260] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.462601 3167 tablet_bootstrap.cc:492] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
W20260927 23:28:12.416033 2763 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 248975us
I20260927 23:28:12.466887 3072 ts_tablet_manager.cc:1428] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.117s user 0.005s sys 0.001s
I20260927 23:28:12.468541 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 114) took 6790 ms. Trace:
I20260927 23:28:12.468892 3336 rpcz_store.cc:276] 0927 23:28:05.678244 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:05.678408 (+ 164us) service_pool.cc:227] Handling call
0927 23:28:11.558809 (+5880401us) ts_tablet_manager.cc:1145] Deleting tablet 74fc2f03baae4f70b38b8079deb8a854
0927 23:28:11.558854 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:11.878475 (+319621us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:12.329298 (+450823us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:12.468468 (+139170us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5842281}
I20260927 23:28:12.469980 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 74fc2f03baae4f70b38b8079deb8a854 (table default.Table_a2512a58a2fe47ff928e54a9b0a8f0f1 [id=052cb2b2fbda4be3825be24c2325b85c]) successfully deleted
I20260927 23:28:12.376739 3749 raft_consensus.cc:482] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:12.335314 2651 tablet_service.cc:1467] Processing CreateTablet for tablet fa65be0c3c4347a48a53fa2a3f6e7544 (DEFAULT_TABLE table=default.Table_59054c732de9495093fd5a97281b4cf5 [id=fe1f8e25be8b40beace6ec0818f026ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.471223 3773 raft_consensus.cc:482] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de216a4a8e4b4e1b857d8647a3b05f6d)
I20260927 23:28:12.471230 3749 raft_consensus.cc:504] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.471694 3773 raft_consensus.cc:504] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.472273 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa65be0c3c4347a48a53fa2a3f6e7544. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.473294 3749 leader_election.cc:288] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:12.425806 2646 tablet_service.cc:1467] Processing CreateTablet for tablet df3a69b419ee4f37a2f6f9bffc7dd2dc (DEFAULT_TABLE table=default.Table_585d1f792a1b42e7ae86d5ecb5edaff2 [id=ff09406f8316449ead4cffa47a2095b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.474481 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40bce5362d2541329b253716bbd99e07" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:12.474897 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df3a69b419ee4f37a2f6f9bffc7dd2dc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:12.475950 2609 leader_election.cc:341] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:12.476058 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40bce5362d2541329b253716bbd99e07" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:28:12.477826 2608 leader_election.cc:341] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:12.478246 2608 leader_election.cc:302] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:12.481433 3749 raft_consensus.cc:2726] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:12.482561 3749 consensus_queue.cc:1035] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:12.491587 3749 consensus_queue.cc:1035] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:12.497547 3070 tablet_bootstrap.cc:700] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent opening tablet: real 0.178s user 0.003s sys 0.000s
I20260927 23:28:12.498283 3070 tablet_bootstrap.cc:654] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:12.502928 2650 tablet_service.cc:1467] Processing CreateTablet for tablet bc9fb7e66b6b45c29688d9379e24d95d (DEFAULT_TABLE table=default.Table_4f6bc587407d4e21a4a4a213a07b0ed8 [id=c54e791a750941d79f74e5a943b8a2b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.504061 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc9fb7e66b6b45c29688d9379e24d95d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.508468 2947 tablet_service.cc:1467] Processing CreateTablet for tablet bc9fb7e66b6b45c29688d9379e24d95d (DEFAULT_TABLE table=default.Table_4f6bc587407d4e21a4a4a213a07b0ed8 [id=c54e791a750941d79f74e5a943b8a2b6]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:12.510326 2378 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 2d0d8a82c5234880ab8396b1777e36c8 consensus state has no leader
I20260927 23:28:12.526744 3773 leader_election.cc:288] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:12.528383 2761 leader_election.cc:341] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: a4277045ec344ad7a48db9b14bea5e7b
W20260927 23:28:12.530454 2763 leader_election.cc:341] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: a4277045ec344ad7a48db9b14bea5e7b
I20260927 23:28:12.530983 2763 leader_election.cc:302] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:12.531522 3713 raft_consensus.cc:482] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:12.531978 3713 raft_consensus.cc:504] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.534003 3713 leader_election.cc:288] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:12.535720 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da38499af164d7fb5c1a8d65167ad24" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:12.538410 2906 leader_election.cc:341] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:12.546034 2649 tablet_service.cc:1467] Processing CreateTablet for tablet 1275905f179e4f7eb7bf906baf8c8e03 (DEFAULT_TABLE table=default.Table_2bf48d7e1578434b879f5c977451801f [id=db42fca72e7443c2b700a9fb46994a0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.547370 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1275905f179e4f7eb7bf906baf8c8e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.547312 3769 raft_consensus.cc:482] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de216a4a8e4b4e1b857d8647a3b05f6d)
I20260927 23:28:12.548156 3773 raft_consensus.cc:2726] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:12.547814 3769 raft_consensus.cc:504] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.548707 2824 raft_consensus.cc:3037] T 92b7eb04f8ff4726b4e9fe3d048ddafc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:12.552435 3769 leader_election.cc:288] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:12.553447 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f530db3b52944f6a283ee9e2bf66828" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:12.555857 3558 raft_consensus.cc:2229] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:12.557381 2647 tablet_service.cc:1467] Processing CreateTablet for tablet 178503861b1648d9a931d1606d807ed8 (DEFAULT_TABLE table=default.Table_63f9354acd7d4ca3ad92ab1edf9d7174 [id=0a5810627a964e7caf66c69e7c3ec9b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.558815 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 178503861b1648d9a931d1606d807ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.570585 3793 consensus_queue.cc:1035] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:12.576776 2932 tablet_service.cc:1467] Processing CreateTablet for tablet 178503861b1648d9a931d1606d807ed8 (DEFAULT_TABLE table=default.Table_63f9354acd7d4ca3ad92ab1edf9d7174 [id=0a5810627a964e7caf66c69e7c3ec9b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.579241 3072 tablet_bootstrap.cc:492] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:12.580837 3558 raft_consensus.cc:2258] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:12.582340 3070 tablet_bootstrap.cc:492] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:12.582762 3070 ts_tablet_manager.cc:1397] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.388s user 0.010s sys 0.000s
I20260927 23:28:12.584091 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f530db3b52944f6a283ee9e2bf66828" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:12.590245 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 178503861b1648d9a931d1606d807ed8 (DEFAULT_TABLE table=default.Table_63f9354acd7d4ca3ad92ab1edf9d7174 [id=0a5810627a964e7caf66c69e7c3ec9b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.593264 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 178503861b1648d9a931d1606d807ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.598994 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da38499af164d7fb5c1a8d65167ad24" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:12.599763 2669 raft_consensus.cc:2454] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
W20260927 23:28:12.600162 2907 consensus_peers.cc:487] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 702ebdeb7c5a4b7d900dbb00236cb8ae. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:12.601516 2907 leader_election.cc:302] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:12.602317 3680 raft_consensus.cc:2781] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:12.602676 3680 raft_consensus.cc:482] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:12.602985 3680 raft_consensus.cc:3037] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:12.606024 2763 leader_election.cc:302] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:12.612478 3769 raft_consensus.cc:2726] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:28:12.613095 2609 consensus_peers.cc:487] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:12.614126 2800 tablet_service.cc:1467] Processing CreateTablet for tablet 1275905f179e4f7eb7bf906baf8c8e03 (DEFAULT_TABLE table=default.Table_2bf48d7e1578434b879f5c977451801f [id=db42fca72e7443c2b700a9fb46994a0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.615384 2800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1275905f179e4f7eb7bf906baf8c8e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.616487 3749 consensus_queue.cc:1035] T 92b7eb04f8ff4726b4e9fe3d048ddafc P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:12.618359 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc9fb7e66b6b45c29688d9379e24d95d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.620738 3797 tablet.cc:1722] T 1ae1e85c32a94f868ef48ad471462b3e P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:12.616183 3680 raft_consensus.cc:504] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.626120 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da38499af164d7fb5c1a8d65167ad24" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:12.628098 2906 leader_election.cc:341] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:12.632355 3680 leader_election.cc:288] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:12.636947 2386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 178503861b1648d9a931d1606d807ed8 consensus state has no leader
I20260927 23:28:12.642165 2392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet fc60d0c7a3794607814393f033238052
I20260927 23:28:12.645679 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da38499af164d7fb5c1a8d65167ad24" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:12.646277 2669 raft_consensus.cc:3037] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:12.648660 2933 tablet_service.cc:1514] Processing DeleteTablet for tablet fc60d0c7a3794607814393f033238052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:12 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:12.649802 3072 tablet_bootstrap.cc:654] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:12.651607 2653 tablet_service.cc:1514] Processing DeleteTablet for tablet fc60d0c7a3794607814393f033238052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:12 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:12.653283 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet fc60d0c7a3794607814393f033238052 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc60d0c7a3794607814393f033238052 already in progress: creating tablet
I20260927 23:28:12.653589 2796 tablet_service.cc:1514] Processing DeleteTablet for tablet fc60d0c7a3794607814393f033238052 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:12 UTC) from {username='slave'} at 127.0.0.1:42492
W20260927 23:28:12.647832 3298 thread.cc:636] rpc worker (service pool kudu.consensus.ConsensusService) Time spent creating pthread: real 0.552s user 0.009s sys 0.006s
W20260927 23:28:12.654651 3298 thread.cc:602] rpc worker (service pool kudu.consensus.ConsensusService) Time spent starting thread: real 0.559s user 0.009s sys 0.007s
I20260927 23:28:12.661329 3558 ts_tablet_manager.cc:1905] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:12.670222 3680 consensus_queue.cc:1035] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:12.674714 2933 tablet_service.cc:1467] Processing CreateTablet for tablet 1275905f179e4f7eb7bf906baf8c8e03 (DEFAULT_TABLE table=default.Table_2bf48d7e1578434b879f5c977451801f [id=db42fca72e7443c2b700a9fb46994a0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.686090 2796 tablet_service.cc:1467] Processing CreateTablet for tablet 792b4f6455da43e3bd148b6b7ea33e7b (DEFAULT_TABLE table=default.Table_e6e912fbd17e426b9eb07d4ea695a57a [id=99e41d61391f4b48820708075ad85bf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.689371 2932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 178503861b1648d9a931d1606d807ed8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.692523 3336 tablet_replica.cc:330] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:12.693432 2933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1275905f179e4f7eb7bf906baf8c8e03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.693158 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_6e2ee819c0404d22b95ad103ebf203e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:12.694823 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_6e2ee819c0404d22b95ad103ebf203e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:12.696904 2386 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_6487eaf832c448fbbc9b1e6622e5f105" } new_table_name: "default.Table_6d5283d4244b4b89a156e5dceea65504" modify_external_catalogs: true
I20260927 23:28:12.698391 3336 raft_consensus.cc:2229] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:12.699139 3336 pending_rounds.cc:62] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d: Trying to abort 1 pending ops.
I20260927 23:28:12.699337 3336 pending_rounds.cc:68] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099728269565952 op_type: NO_OP noop_request { }
I20260927 23:28:12.699685 3336 raft_consensus.cc:2866] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:28:12.700006 3336 raft_consensus.cc:2258] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:12.691859 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 792b4f6455da43e3bd148b6b7ea33e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.709355 2669 raft_consensus.cc:2454] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:12.710017 3336 ts_tablet_manager.cc:1905] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:12.710596 2907 leader_election.cc:302] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:12.712520 3680 raft_consensus.cc:2781] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:12.738724 3791 consensus_queue.cc:1035] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:12.742794 2653 tablet_service.cc:1467] Processing CreateTablet for tablet 792b4f6455da43e3bd148b6b7ea33e7b (DEFAULT_TABLE table=default.Table_e6e912fbd17e426b9eb07d4ea695a57a [id=99e41d61391f4b48820708075ad85bf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.743937 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 792b4f6455da43e3bd148b6b7ea33e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.744557 3072 tablet_bootstrap.cc:492] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:12.744988 3072 ts_tablet_manager.cc:1397] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.166s user 0.011s sys 0.000s
I20260927 23:28:12.712965 3680 raft_consensus.cc:686] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:12.751297 3680 consensus_queue.cc:237] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.747323 3072 raft_consensus.cc:348] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.755323 3072 raft_consensus.cc:374] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:12.755579 3072 raft_consensus.cc:729] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:12.691675 3070 raft_consensus.cc:348] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.757881 3070 raft_consensus.cc:374] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:12.758332 3070 raft_consensus.cc:729] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:12.759104 3070 consensus_queue.cc:260] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.756146 3072 consensus_queue.cc:260] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.761793 3070 ts_tablet_manager.cc:1428] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.179s user 0.007s sys 0.000s
I20260927 23:28:12.776060 3072 ts_tablet_manager.cc:1428] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.031s user 0.006s sys 0.001s
I20260927 23:28:12.777890 3072 tablet_bootstrap.cc:492] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:12.700340 3167 tablet_bootstrap.cc:654] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:12.684727 3800 tablet.cc:1722] T 1ae1e85c32a94f868ef48ad471462b3e P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:12.793473 3336 ts_tablet_manager.cc:1918] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:12.793918 3336 log.cc:1196] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/702ebdeb7c5a4b7d900dbb00236cb8ae
I20260927 23:28:12.794641 3336 ts_tablet_manager.cc:1939] T 702ebdeb7c5a4b7d900dbb00236cb8ae P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:12.715837 3802 tablet.cc:1722] T 1ae1e85c32a94f868ef48ad471462b3e P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:12.804215 2807 tablet_service.cc:1467] Processing CreateTablet for tablet 827658d06ff24ad8b1f6b5039d0a5847 (DEFAULT_TABLE table=default.Table_65cd17e1885c4ba6b11fb71b7c19ab4b [id=19c0ff588ce344e79d73f0424ad2f876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.805105 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 116) took 6840 ms. Trace:
I20260927 23:28:12.805769 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827658d06ff24ad8b1f6b5039d0a5847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.805397 2933 tablet_service.cc:1467] Processing CreateTablet for tablet 792b4f6455da43e3bd148b6b7ea33e7b (DEFAULT_TABLE table=default.Table_e6e912fbd17e426b9eb07d4ea695a57a [id=99e41d61391f4b48820708075ad85bf8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.805464 3336 rpcz_store.cc:276] 0927 23:28:05.964933 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:05.965025 (+ 92us) service_pool.cc:227] Handling call
0927 23:28:12.593521 (+6628496us) ts_tablet_manager.cc:1145] Deleting tablet 702ebdeb7c5a4b7d900dbb00236cb8ae
0927 23:28:12.692360 (+ 98839us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:12.738167 (+ 45807us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:12.771115 (+ 32948us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:12.805033 (+ 33918us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5031,"tablet-delete.queue_time_us":6627983}
I20260927 23:28:12.807839 2933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 792b4f6455da43e3bd148b6b7ea33e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.808096 3336 tablet_replica.cc:330] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:12.818259 3336 raft_consensus.cc:2229] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:12.821935 3336 raft_consensus.cc:2258] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:12.830127 3336 ts_tablet_manager.cc:1905] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:12.822829 3167 tablet_bootstrap.cc:492] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:12.834944 3167 ts_tablet_manager.cc:1397] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.373s user 0.010s sys 0.001s
I20260927 23:28:12.834965 3572 consensus_queue.cc:1035] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:12.836460 3072 tablet_bootstrap.cc:654] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:12.836160 2798 tablet_service.cc:1467] Processing CreateTablet for tablet ad8cdfaefc094c2394eb87f6bf33399e (DEFAULT_TABLE table=default.Table_f580f5603b1b4c689ae82d0303ad8c4b [id=89124bc4ca2c4d37b3d4c0baa7af83a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.845642 2798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8cdfaefc094c2394eb87f6bf33399e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.785234 2644 tablet_service.cc:1467] Processing CreateTablet for tablet 827658d06ff24ad8b1f6b5039d0a5847 (DEFAULT_TABLE table=default.Table_65cd17e1885c4ba6b11fb71b7c19ab4b [id=19c0ff588ce344e79d73f0424ad2f876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.860201 2644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827658d06ff24ad8b1f6b5039d0a5847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.870913 2932 tablet_service.cc:1467] Processing CreateTablet for tablet 827658d06ff24ad8b1f6b5039d0a5847 (DEFAULT_TABLE table=default.Table_65cd17e1885c4ba6b11fb71b7c19ab4b [id=19c0ff588ce344e79d73f0424ad2f876]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.872145 2932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 827658d06ff24ad8b1f6b5039d0a5847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.881492 3070 tablet_bootstrap.cc:492] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
W20260927 23:28:12.885322 2906 consensus_peers.cc:487] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:12.926802 3167 raft_consensus.cc:348] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.927758 3167 raft_consensus.cc:374] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:12.928009 3167 raft_consensus.cc:729] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:12.928640 2388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 10fe6b0a4c614b77ae95b200b6a12843
I20260927 23:28:12.930009 2388 catalog_manager.cc:6565] ReplaceTablet: tablet 10fe6b0a4c614b77ae95b200b6a12843 deleted and replaced by tablet 46e23759c43c49ba85a3ba8c47131850
I20260927 23:28:12.937120 2794 tablet_service.cc:1514] Processing DeleteTablet for tablet 10fe6b0a4c614b77ae95b200b6a12843 with delete_type TABLET_DATA_DELETED (replaced by tablet 46e23759c43c49ba85a3ba8c47131850) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:12.959872 2646 tablet_service.cc:1514] Processing DeleteTablet for tablet 10fe6b0a4c614b77ae95b200b6a12843 with delete_type TABLET_DATA_DELETED (replaced by tablet 46e23759c43c49ba85a3ba8c47131850) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:12.961901 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 10fe6b0a4c614b77ae95b200b6a12843 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 10fe6b0a4c614b77ae95b200b6a12843 already in progress: creating tablet
I20260927 23:28:12.967370 3167 consensus_queue.cc:260] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:12.973402 2646 tablet_service.cc:1467] Processing CreateTablet for tablet ad8cdfaefc094c2394eb87f6bf33399e (DEFAULT_TABLE table=default.Table_f580f5603b1b4c689ae82d0303ad8c4b [id=89124bc4ca2c4d37b3d4c0baa7af83a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:12.974576 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8cdfaefc094c2394eb87f6bf33399e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:12.984557 2943 tablet_service.cc:1514] Processing DeleteTablet for tablet 10fe6b0a4c614b77ae95b200b6a12843 with delete_type TABLET_DATA_DELETED (replaced by tablet 46e23759c43c49ba85a3ba8c47131850) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:13.009315 3167 ts_tablet_manager.cc:1428] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.174s user 0.007s sys 0.000s
I20260927 23:28:13.026011 3167 tablet_bootstrap.cc:492] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:13.033043 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 5e904dcd585c4849a18b3b3e11e9e4bf (DEFAULT_TABLE table=default.Table_849b837949944562bb32d87e0bafbb48 [id=7000696501b84eaa88962c5cc45ed57c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:13.034559 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e904dcd585c4849a18b3b3e11e9e4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:13.046147 3336 ts_tablet_manager.cc:1918] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:13.046612 3336 log.cc:1196] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/cc1c0da6ea5c4a09a71555f37becae33
I20260927 23:28:13.047344 3336 ts_tablet_manager.cc:1939] T cc1c0da6ea5c4a09a71555f37becae33 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:13.049917 3796 consensus_queue.cc:1035] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:13.052327 3167 tablet_bootstrap.cc:654] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:13.060853 3558 ts_tablet_manager.cc:1918] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:13.060845 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 117) took 6398 ms. Trace:
I20260927 23:28:13.061184 3336 rpcz_store.cc:276] 0927 23:28:06.662561 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.662698 (+ 137us) service_pool.cc:227] Handling call
0927 23:28:12.807941 (+6145243us) ts_tablet_manager.cc:1145] Deleting tablet cc1c0da6ea5c4a09a71555f37becae33
0927 23:28:12.807988 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:12.864723 (+ 56735us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.045893 (+181170us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.060745 (+ 14852us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":6144655}
I20260927 23:28:13.061247 3558 log.cc:1196] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/a4277045ec344ad7a48db9b14bea5e7b
I20260927 23:28:13.064667 3558 ts_tablet_manager.cc:1939] T a4277045ec344ad7a48db9b14bea5e7b P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:13.063673 2943 tablet_service.cc:1467] Processing CreateTablet for tablet ad8cdfaefc094c2394eb87f6bf33399e (DEFAULT_TABLE table=default.Table_f580f5603b1b4c689ae82d0303ad8c4b [id=89124bc4ca2c4d37b3d4c0baa7af83a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:13.065702 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad8cdfaefc094c2394eb87f6bf33399e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:13.066811 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 152) took 4294 ms. Trace:
I20260927 23:28:13.067085 3558 rpcz_store.cc:276] 0927 23:28:08.772503 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.772651 (+ 148us) service_pool.cc:227] Handling call
0927 23:28:12.390502 (+3617851us) ts_tablet_manager.cc:1145] Deleting tablet a4277045ec344ad7a48db9b14bea5e7b
0927 23:28:12.390559 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:12.854684 (+464125us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.060563 (+205879us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.066733 (+ 6170us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":43460,"tablet-delete.queue_time_us":3598419}
I20260927 23:28:13.068468 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet a4277045ec344ad7a48db9b14bea5e7b (table default.Table_a902e331f690448a872f6c104857f2ca [id=1acae7217ece44919a8d247f5cf155d1]) successfully deleted
I20260927 23:28:13.077677 3336 tablet_replica.cc:330] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:13.078634 3336 raft_consensus.cc:2229] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:13.079922 3336 raft_consensus.cc:2258] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:13.081571 3072 tablet_bootstrap.cc:492] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:13.082021 3072 ts_tablet_manager.cc:1397] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.304s user 0.005s sys 0.013s
I20260927 23:28:13.085101 3072 raft_consensus.cc:348] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.085933 3072 raft_consensus.cc:374] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:13.086119 3167 tablet_bootstrap.cc:492] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:13.086177 3072 raft_consensus.cc:729] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:13.086676 3167 ts_tablet_manager.cc:1397] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.061s user 0.007s sys 0.008s
I20260927 23:28:13.086997 3336 ts_tablet_manager.cc:1905] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:13.086875 3072 consensus_queue.cc:260] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.088954 3072 ts_tablet_manager.cc:1428] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260927 23:28:13.089908 3072 tablet_bootstrap.cc:492] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
W20260927 23:28:13.090008 2609 consensus_peers.cc:487] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:13.096696 2804 tablet_service.cc:1467] Processing CreateTablet for tablet ae5e42d0292c47738def100f2822f847 (DEFAULT_TABLE table=default.Table_9b16095aa7824f10a46040ac08836015 [id=157e4b5315e047478ba67e506794f77a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:13.097795 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae5e42d0292c47738def100f2822f847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:13.098112 2649 tablet_service.cc:1467] Processing CreateTablet for tablet ae5e42d0292c47738def100f2822f847 (DEFAULT_TABLE table=default.Table_9b16095aa7824f10a46040ac08836015 [id=157e4b5315e047478ba67e506794f77a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:13.099157 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae5e42d0292c47738def100f2822f847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:13.091300 3167 raft_consensus.cc:348] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.100988 3167 raft_consensus.cc:374] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:13.101233 3167 raft_consensus.cc:729] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:13.104812 3558 tablet_replica.cc:330] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:13.105611 3558 raft_consensus.cc:2229] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:13.107851 3558 pending_rounds.cc:62] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51: Trying to abort 1 pending ops.
I20260927 23:28:13.108609 3167 consensus_queue.cc:260] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.111862 3167 ts_tablet_manager.cc:1428] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.025s user 0.001s sys 0.004s
I20260927 23:28:13.112794 3167 tablet_bootstrap.cc:492] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:13.114326 3070 tablet_bootstrap.cc:654] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:13.108063 3558 pending_rounds.cc:68] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099716800065536 op_type: NO_OP noop_request { }
I20260927 23:28:13.114832 3558 raft_consensus.cc:2866] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:28:13.115233 3558 raft_consensus.cc:2258] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:13.122233 2389 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 9af9b87abf1f4f78b3140a91f2525066
I20260927 23:28:13.123942 3769 raft_consensus.cc:482] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:13.124459 3769 raft_consensus.cc:504] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.126417 3769 leader_election.cc:288] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:13.127080 2796 tablet_service.cc:1514] Processing DeleteTablet for tablet 9af9b87abf1f4f78b3140a91f2525066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:12 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:13.128055 2652 tablet_service.cc:1514] Processing DeleteTablet for tablet 9af9b87abf1f4f78b3140a91f2525066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:12 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:13.128274 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2a39980f0c46c68becefacad890ab7" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:13.129851 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 9af9b87abf1f4f78b3140a91f2525066 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9af9b87abf1f4f78b3140a91f2525066 already in progress: creating tablet
I20260927 23:28:13.130712 3336 ts_tablet_manager.cc:1918] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:13.131078 3336 log.cc:1196] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/2d837daee7a04b338813519acf53dfae
I20260927 23:28:13.133739 3336 ts_tablet_manager.cc:1939] T 2d837daee7a04b338813519acf53dfae P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:13.135871 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 118) took 6289 ms. Trace:
I20260927 23:28:13.136159 3336 rpcz_store.cc:276] 0927 23:28:06.846495 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.846601 (+ 106us) service_pool.cc:227] Handling call
0927 23:28:13.065180 (+6218579us) ts_tablet_manager.cc:1145] Deleting tablet 2d837daee7a04b338813519acf53dfae
0927 23:28:13.077526 (+ 12346us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.121821 (+ 44295us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.130448 (+ 8627us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.135806 (+ 5358us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3328,"tablet-delete.queue_time_us":6218028}
I20260927 23:28:13.137256 3336 tablet_replica.cc:330] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:13.137980 3336 raft_consensus.cc:2229] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:13.138891 3336 raft_consensus.cc:2258] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:13.140796 3336 ts_tablet_manager.cc:1905] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:13.144677 3769 raft_consensus.cc:482] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:13.156054 3769 raft_consensus.cc:504] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.158136 3769 leader_election.cc:288] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:13.158633 3336 ts_tablet_manager.cc:1918] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:13.158958 3336 log.cc:1196] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/4929fc9e396e48a189c94586d58aa55f
I20260927 23:28:13.159624 3336 ts_tablet_manager.cc:1939] T 4929fc9e396e48a189c94586d58aa55f P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:13.163234 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da38499af164d7fb5c1a8d65167ad24" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:13.164067 2669 raft_consensus.cc:2379] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate de216a4a8e4b4e1b857d8647a3b05f6d in this term.
I20260927 23:28:13.166277 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 119) took 6220 ms. Trace:
I20260927 23:28:13.166601 3336 rpcz_store.cc:276] 0927 23:28:06.945415 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:06.945535 (+ 120us) service_pool.cc:227] Handling call
0927 23:28:13.137091 (+6191556us) ts_tablet_manager.cc:1145] Deleting tablet 4929fc9e396e48a189c94586d58aa55f
0927 23:28:13.137133 (+ 42us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.149807 (+ 12674us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.158396 (+ 8589us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.166195 (+ 7799us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4470,"tablet-delete.queue_time_us":6172888}
I20260927 23:28:13.167766 3167 tablet_bootstrap.cc:654] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:13.167852 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 137) took 5024 ms. Trace:
I20260927 23:28:13.168123 3336 rpcz_store.cc:276] 0927 23:28:08.143423 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.146305 (+ 2882us) service_pool.cc:227] Handling call
0927 23:28:13.167550 (+5021245us) ts_tablet_manager.cc:1145] Deleting tablet 95216fdc2e104a2e9fc4bc424bdbe5e9
0927 23:28:13.167594 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.167796 (+ 202us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5020618}
I20260927 23:28:13.169201 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 140) took 4882 ms. Trace:
I20260927 23:28:13.169497 3336 rpcz_store.cc:276] 0927 23:28:08.286950 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.287451 (+ 501us) service_pool.cc:227] Handling call
0927 23:28:13.168944 (+4881493us) ts_tablet_manager.cc:1145] Deleting tablet 45dd55b5a9d94ff0b84a799174ab5359
0927 23:28:13.168982 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.169150 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4880916}
I20260927 23:28:13.170459 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 142) took 4835 ms. Trace:
I20260927 23:28:13.170733 3336 rpcz_store.cc:276] 0927 23:28:08.334839 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.334938 (+ 99us) service_pool.cc:227] Handling call
0927 23:28:13.170209 (+4835271us) ts_tablet_manager.cc:1145] Deleting tablet 9cce47b050b445dfba7be5ea3b6a8636
0927 23:28:13.170245 (+ 36us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.170417 (+ 172us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4815218}
I20260927 23:28:13.171638 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 143) took 4836 ms. Trace:
I20260927 23:28:13.174175 3558 ts_tablet_manager.cc:1905] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:13.180202 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet cc1c0da6ea5c4a09a71555f37becae33 (table default.Table_29404653bfbb4e5d8192bcb604084df0 [id=baad95258e1a4358a58abc3fc1cc3737]) successfully deleted
I20260927 23:28:13.182971 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 4929fc9e396e48a189c94586d58aa55f (table default.Table_db0f806a353a4dbdb9d64635383bc826 [id=f0d0efa2f91248b784bb2ab87b87fbf1]) successfully deleted
W20260927 23:28:13.184135 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 95216fdc2e104a2e9fc4bc424bdbe5e9 because the tablet was not found. No further retry: Not found: Tablet not found: 95216fdc2e104a2e9fc4bc424bdbe5e9
I20260927 23:28:13.190176 3167 tablet_bootstrap.cc:492] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:13.190645 3167 ts_tablet_manager.cc:1397] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.078s user 0.005s sys 0.003s
I20260927 23:28:13.171886 3336 rpcz_store.cc:276] 0927 23:28:08.335587 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.335682 (+ 95us) service_pool.cc:227] Handling call
0927 23:28:13.171426 (+4835744us) ts_tablet_manager.cc:1145] Deleting tablet 66f2842218fc42e5b1b83314e5f1a158
0927 23:28:13.171461 (+ 35us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.171597 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":68,"tablet-delete.queue_time_us":4806880}
I20260927 23:28:13.197229 3167 raft_consensus.cc:348] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.197923 3167 raft_consensus.cc:374] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:13.198171 3167 raft_consensus.cc:729] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:13.198736 3167 consensus_queue.cc:260] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.197947 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 141) took 4910 ms. Trace:
I20260927 23:28:13.204913 3336 rpcz_store.cc:276] 0927 23:28:08.287217 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.334134 (+ 46917us) service_pool.cc:227] Handling call
0927 23:28:13.197640 (+4863506us) ts_tablet_manager.cc:1145] Deleting tablet a80bff324902498fa5c04fe8cbc05f9d
0927 23:28:13.197686 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.197887 (+ 201us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2233,"tablet-delete.queue_time_us":4832890}
I20260927 23:28:13.209887 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 144) took 4873 ms. Trace:
I20260927 23:28:13.210233 3336 rpcz_store.cc:276] 0927 23:28:08.336320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.353719 (+ 17399us) service_pool.cc:227] Handling call
0927 23:28:13.209597 (+4855878us) ts_tablet_manager.cc:1145] Deleting tablet 5efbd0211cf3408cb5ea9640d8da6694
0927 23:28:13.209645 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.209827 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8662,"tablet-delete.queue_time_us":4843537}
I20260927 23:28:13.211382 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 146) took 4845 ms. Trace:
I20260927 23:28:13.211692 3336 rpcz_store.cc:276] 0927 23:28:08.365806 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.412763 (+ 46957us) service_pool.cc:227] Handling call
0927 23:28:13.211116 (+4798353us) ts_tablet_manager.cc:1145] Deleting tablet 7767a32c07454fe6a92154429a3b6899
0927 23:28:13.211153 (+ 37us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.211332 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4797698}
I20260927 23:28:13.213725 2653 tablet_service.cc:1467] Processing CreateTablet for tablet 5e904dcd585c4849a18b3b3e11e9e4bf (DEFAULT_TABLE table=default.Table_849b837949944562bb32d87e0bafbb48 [id=7000696501b84eaa88962c5cc45ed57c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:13.214857 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e904dcd585c4849a18b3b3e11e9e4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:13.216177 3167 ts_tablet_manager.cc:1428] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.025s user 0.007s sys 0.000s
W20260927 23:28:13.217479 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 45dd55b5a9d94ff0b84a799174ab5359 because the tablet was not found. No further retry: Not found: Tablet not found: 45dd55b5a9d94ff0b84a799174ab5359
W20260927 23:28:13.218415 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 9cce47b050b445dfba7be5ea3b6a8636 because the tablet was not found. No further retry: Not found: Tablet not found: 9cce47b050b445dfba7be5ea3b6a8636
I20260927 23:28:13.219933 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e2a39980f0c46c68becefacad890ab7" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:13.221534 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da38499af164d7fb5c1a8d65167ad24" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:13.223415 2947 tablet_service.cc:1467] Processing CreateTablet for tablet 5e904dcd585c4849a18b3b3e11e9e4bf (DEFAULT_TABLE table=default.Table_849b837949944562bb32d87e0bafbb48 [id=7000696501b84eaa88962c5cc45ed57c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:13.224511 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e904dcd585c4849a18b3b3e11e9e4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:13.228631 3167 tablet_bootstrap.cc:492] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:13.229079 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 139) took 4941 ms. Trace:
I20260927 23:28:13.229415 3336 rpcz_store.cc:276] 0927 23:28:08.286680 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.397088 (+110408us) service_pool.cc:227] Handling call
0927 23:28:13.228001 (+4830913us) ts_tablet_manager.cc:1145] Deleting tablet 828f870365244777b59e8a0756d7d4c9
0927 23:28:13.228048 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.228229 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4814218}
I20260927 23:28:13.230530 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 145) took 4875 ms. Trace:
I20260927 23:28:13.230799 3336 rpcz_store.cc:276] 0927 23:28:08.354659 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.355227 (+ 568us) service_pool.cc:227] Handling call
0927 23:28:13.230224 (+4874997us) ts_tablet_manager.cc:1145] Deleting tablet 7096e434ac9f4290ad58ba4f7e7c2513
0927 23:28:13.230277 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.230474 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4808459}
I20260927 23:28:13.231740 3336 tablet_replica.cc:330] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:13.233081 2943 tablet_service.cc:1514] Processing DeleteTablet for tablet 9af9b87abf1f4f78b3140a91f2525066 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:12 UTC) from {username='slave'} at 127.0.0.1:44952
W20260927 23:28:13.234869 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 66f2842218fc42e5b1b83314e5f1a158 because the tablet was not found. No further retry: Not found: Tablet not found: 66f2842218fc42e5b1b83314e5f1a158
W20260927 23:28:13.235831 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet a80bff324902498fa5c04fe8cbc05f9d because the tablet was not found. No further retry: Not found: Tablet not found: a80bff324902498fa5c04fe8cbc05f9d
I20260927 23:28:13.246057 2798 tablet_service.cc:1467] Processing CreateTablet for tablet 35892219ae7140e58c96707d08190d4d (DEFAULT_TABLE table=default.Table_2ed30e794ca8476ea9111e99393a0371 [id=8650cd807b99452ab96a59cea71c8e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:13.247411 2798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35892219ae7140e58c96707d08190d4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:13.247958 3769 raft_consensus.cc:482] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:13.248484 3769 raft_consensus.cc:504] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.249652 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:13.250578 3769 leader_election.cc:288] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:13.250716 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:13.251737 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:13.253808 3336 raft_consensus.cc:2229] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:13.253260 2388 catalog_manager.cc:5495] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } }
I20260927 23:28:13.254349 3336 raft_consensus.cc:2258] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:13.256139 2763 leader_election.cc:341] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: ac19099dd7bf4b669d504a654ee69482
I20260927 23:28:13.257637 3336 ts_tablet_manager.cc:1905] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:13.267623 3167 tablet_bootstrap.cc:654] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:13.275620 3558 ts_tablet_manager.cc:1918] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:13.276333 2824 raft_consensus.cc:3037] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:13.300601 3072 tablet_bootstrap.cc:654] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:13.306727 3558 log.cc:1196] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/cfbbe17f54d1418b95efc3943db0b6c1
I20260927 23:28:13.308959 3558 ts_tablet_manager.cc:1939] T cfbbe17f54d1418b95efc3943db0b6c1 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:13.312433 2943 tablet_service.cc:1467] Processing CreateTablet for tablet ae5e42d0292c47738def100f2822f847 (DEFAULT_TABLE table=default.Table_9b16095aa7824f10a46040ac08836015 [id=157e4b5315e047478ba67e506794f77a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:13.314157 2933 tablet_service.cc:1467] Processing CreateTablet for tablet 35892219ae7140e58c96707d08190d4d (DEFAULT_TABLE table=default.Table_2ed30e794ca8476ea9111e99393a0371 [id=8650cd807b99452ab96a59cea71c8e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:13.308844 2761 leader_election.cc:302] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:13.315577 2761 leader_election.cc:302] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:13.316366 3167 tablet_bootstrap.cc:492] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:13.316795 3167 ts_tablet_manager.cc:1397] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.088s user 0.010s sys 0.000s
I20260927 23:28:13.319345 3167 raft_consensus.cc:348] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.320056 3167 raft_consensus.cc:374] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260927 23:28:13.307991 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 5efbd0211cf3408cb5ea9640d8da6694 because the tablet was not found. No further retry: Not found: Tablet not found: 5efbd0211cf3408cb5ea9640d8da6694
W20260927 23:28:13.322383 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 7767a32c07454fe6a92154429a3b6899 because the tablet was not found. No further retry: Not found: Tablet not found: 7767a32c07454fe6a92154429a3b6899
W20260927 23:28:13.323307 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 7096e434ac9f4290ad58ba4f7e7c2513 because the tablet was not found. No further retry: Not found: Tablet not found: 7096e434ac9f4290ad58ba4f7e7c2513
I20260927 23:28:13.291236 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 35892219ae7140e58c96707d08190d4d (DEFAULT_TABLE table=default.Table_2ed30e794ca8476ea9111e99393a0371 [id=8650cd807b99452ab96a59cea71c8e4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:13.324050 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 1 got 0
I20260927 23:28:13.329348 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35892219ae7140e58c96707d08190d4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:13.329373 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1da38499af164d7fb5c1a8d65167ad24 (table default.Table_f7b5c90ef9a94341a88b0a40ea0f7e1e [id=b12aae65f933476ab57974d4b328a266]). Expected version 3 got 0
I20260927 23:28:13.331025 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 153) took 4533 ms. Trace:
I20260927 23:28:13.342214 3558 rpcz_store.cc:276] 0927 23:28:08.797921 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.800519 (+ 2598us) service_pool.cc:227] Handling call
0927 23:28:13.104612 (+4304093us) ts_tablet_manager.cc:1145] Deleting tablet cfbbe17f54d1418b95efc3943db0b6c1
0927 23:28:13.104688 (+ 76us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.219075 (+114387us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.275378 (+ 56303us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.330934 (+ 55556us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4303495}
I20260927 23:28:13.343060 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ae5e42d0292c47738def100f2822f847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:13.348519 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet cfbbe17f54d1418b95efc3943db0b6c1 (table default.Table_96276d84ee11493d89908e4ff16dc7fe [id=f21b53d8e90a48bc9cb7f6268c946cc1]) successfully deleted
W20260927 23:28:13.349084 2386 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet e795a6d7eee140b181c2846ee5c43e24 consensus state has no leader
I20260927 23:28:13.349068 2933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35892219ae7140e58c96707d08190d4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:13.335592 3840 raft_consensus.cc:3037] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:13.334784 2761 leader_election.cc:341] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: ac19099dd7bf4b669d504a654ee69482
I20260927 23:28:13.352595 2761 leader_election.cc:302] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:13.353261 3167 raft_consensus.cc:729] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:13.354008 3167 consensus_queue.cc:260] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.359254 3558 tablet_replica.cc:330] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:13.366003 3558 raft_consensus.cc:2229] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:13.361387 3843 raft_consensus.cc:2726] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:13.367210 3558 raft_consensus.cc:2258] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:13.369451 3070 tablet_bootstrap.cc:492] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:13.370288 3070 ts_tablet_manager.cc:1397] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.489s user 0.006s sys 0.004s
I20260927 23:28:13.373399 3070 raft_consensus.cc:348] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.374238 3070 raft_consensus.cc:374] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:13.374544 3070 raft_consensus.cc:729] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:13.376952 3336 ts_tablet_manager.cc:1918] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:13.377382 3336 log.cc:1196] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/2411d7c53a154fafad9edda9fa75bb36
I20260927 23:28:13.378113 3336 ts_tablet_manager.cc:1939] T 2411d7c53a154fafad9edda9fa75bb36 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:13.375188 3558 ts_tablet_manager.cc:1905] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:13.388065 3167 ts_tablet_manager.cc:1428] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.071s user 0.008s sys 0.000s
I20260927 23:28:13.388427 3769 raft_consensus.cc:2726] T 0e2a39980f0c46c68becefacad890ab7 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:13.390054 2820 raft_consensus.cc:3037] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:13.396822 3769 raft_consensus.cc:482] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:13.397071 3167 tablet_bootstrap.cc:492] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:13.397289 3769 raft_consensus.cc:504] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.388460 3072 tablet_bootstrap.cc:492] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:13.399151 3769 leader_election.cc:288] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:13.399710 3072 ts_tablet_manager.cc:1397] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.310s user 0.009s sys 0.001s
I20260927 23:28:13.402063 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0d8a82c5234880ab8396b1777e36c8" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:13.405265 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 1 got 0
I20260927 23:28:13.406276 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:13.406988 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1ae1e85c32a94f868ef48ad471462b3e (table default.Table_48ee3ed6e26d4e41aa46aaf16ff3a09d [id=21118fc25eb9485b92a9034bbeef8b13]). Expected version 1 got 0
I20260927 23:28:13.407312 3840 raft_consensus.cc:2726] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:13.411504 3167 tablet_bootstrap.cc:654] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:13.413650 2391 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 1 got 0
I20260927 23:28:13.422530 2389 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 1 got 0
I20260927 23:28:13.423882 2389 catalog_manager.cc:5495] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: HEALTHY } } }
I20260927 23:28:13.425382 2389 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 1da38499af164d7fb5c1a8d65167ad24 (table default.Table_f7b5c90ef9a94341a88b0a40ea0f7e1e [id=b12aae65f933476ab57974d4b328a266]). Expected version 3 got 0
I20260927 23:28:13.426671 2389 catalog_manager.cc:5495] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: HEALTHY } } }
I20260927 23:28:13.428676 2389 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:13.439061 3167 tablet_bootstrap.cc:492] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:13.439509 3167 ts_tablet_manager.cc:1397] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.043s user 0.004s sys 0.006s
I20260927 23:28:13.434504 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_f6b1317b23f04156946dfcd7dc25c076"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:13.440790 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f6b1317b23f04156946dfcd7dc25c076 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:13.436178 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_d72c914ddbb24ee8ba380dc1d49175f0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:13.442291 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d72c914ddbb24ee8ba380dc1d49175f0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:13.442004 3167 raft_consensus.cc:348] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.442644 3167 raft_consensus.cc:374] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:13.442889 3167 raft_consensus.cc:729] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:13.446389 2389 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af]). Expected version 2 got 0
I20260927 23:28:13.447974 3070 consensus_queue.cc:260] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.458804 3070 ts_tablet_manager.cc:1428] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.088s user 0.005s sys 0.000s
I20260927 23:28:13.455582 2389 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:13.474126 2390 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_41e19fe9cdcf42de870775c087f52478 [id=c8c0a7367a8342afb8ffc95627013a9c] alter complete (version 1)
W20260927 23:28:13.476007 2390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 44) took 2928 ms (client timeout 1000 ms). Trace:
I20260927 23:28:13.476558 3558 ts_tablet_manager.cc:1918] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:13.476938 3558 log.cc:1196] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/04f670db808c4e289c655679da9d2d9d
I20260927 23:28:13.477650 3558 ts_tablet_manager.cc:1939] T 04f670db808c4e289c655679da9d2d9d P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:13.479650 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 151) took 4733 ms. Trace:
I20260927 23:28:13.479940 3558 rpcz_store.cc:276] 0927 23:28:08.746055 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.786877 (+ 40822us) service_pool.cc:227] Handling call
0927 23:28:13.359028 (+4572151us) ts_tablet_manager.cc:1145] Deleting tablet 04f670db808c4e289c655679da9d2d9d
0927 23:28:13.359092 (+ 64us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.434823 (+ 75731us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.476289 (+ 41466us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.479589 (+ 3300us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4555490}
I20260927 23:28:13.480921 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet 3200d6bd5cb24ff5b1bbd59af9353dbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:03 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:13.481437 3167 consensus_queue.cc:260] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.487684 3070 tablet_bootstrap.cc:492] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:13.496549 3167 ts_tablet_manager.cc:1428] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.057s user 0.007s sys 0.001s
I20260927 23:28:13.497632 3167 tablet_bootstrap.cc:492] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:13.497690 3558 tablet_replica.cc:330] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:13.505259 3558 raft_consensus.cc:2229] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:13.507138 2646 tablet_service.cc:1514] Processing DeleteTablet for tablet ac19099dd7bf4b669d504a654ee69482 with delete_type TABLET_DATA_DELETED (replaced by tablet 9eb27269a590476f878ee52a2aefd141) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:13.507015 3558 raft_consensus.cc:2258] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:13.515354 3558 ts_tablet_manager.cc:1905] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:13.516167 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet 7767a32c07454fe6a92154429a3b6899 with delete_type TABLET_DATA_DELETED (replaced by tablet 49883297d2b046b99f2ec1343c276ff9) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:13.481199 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 04f670db808c4e289c655679da9d2d9d (table default.Table_9de2a295d1fc4fa692d084f283b81ba8 [id=bd8e999821b44f628e5ec3b1d7915773]) successfully deleted
W20260927 23:28:13.482857 2385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 45) took 1921 ms (client timeout 1000 ms). Trace:
W20260927 23:28:13.519955 2385 rpcz_store.cc:269] 0927 23:28:11.561111 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:11.571689 (+ 10578us) service_pool.cc:227] Handling call
0927 23:28:11.674369 (+102680us) spinlock_profiling.cc:243] Waited 92.8 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:13.482779 (+1808410us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":9,"spinlock_wait_cycles":213468032}
W20260927 23:28:13.508704 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet ac19099dd7bf4b669d504a654ee69482 because the tablet was not found. No further retry: Not found: Tablet not found: ac19099dd7bf4b669d504a654ee69482
W20260927 23:28:13.509825 2387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 31) took 2881 ms (client timeout 1000 ms). Trace:
W20260927 23:28:13.522377 2387 rpcz_store.cc:269] 0927 23:28:10.628881 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.784606 (+155725us) service_pool.cc:227] Handling call
0927 23:28:13.509751 (+2725145us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":42629,"reported_tablets":10}
W20260927 23:28:13.522819 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 7767a32c07454fe6a92154429a3b6899 because the tablet was not found. No further retry: Not found: Tablet not found: 7767a32c07454fe6a92154429a3b6899
I20260927 23:28:13.523600 2385 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_7113798bfe2c4627ac0f51de8a85e8d2" } new_table_name: "default.Table_5a7ac75fad65481099ea0a6956abafd7" modify_external_catalogs: true
I20260927 23:28:13.524362 2387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_eaa0db537a9f4c38b1c0db99654a6292" } modify_external_catalogs: true
W20260927 23:28:13.495249 2391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 28) took 1578 ms (client timeout 1000 ms). Trace:
W20260927 23:28:13.525120 2391 rpcz_store.cc:269] 0927 23:28:11.917014 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:11.955952 (+ 38938us) service_pool.cc:227] Handling call
0927 23:28:12.113142 (+157190us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:13.495168 (+1382026us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":9,"spinlock_wait_cycles":261752704}
I20260927 23:28:13.527458 2798 tablet_service.cc:1514] Processing DeleteTablet for tablet cfbbe17f54d1418b95efc3943db0b6c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:13.528685 2798 tablet_service.cc:1514] Processing DeleteTablet for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae with delete_type TABLET_DATA_DELETED (replaced by tablet 8fadaafd49b04b9483a6a50f2f7d5be7) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:13.530354 3070 tablet_bootstrap.cc:654] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:13.534296 2391 master_service.cc:946] ReplaceTablet: received request to replace tablet 7a5274b56b814c029e02c16243a3e663 from {username='slave'} at 127.0.0.1:51972
W20260927 23:28:13.495724 2390 rpcz_store.cc:269] 0927 23:28:10.547759 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.557643 (+ 9884us) service_pool.cc:227] Handling call
0927 23:28:10.697209 (+139566us) spinlock_profiling.cc:243] Waited 79 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:13.475936 (+2778727us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:10.883237 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:10.883587 (+ 350us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:10.883608 (+ 21us) write_op.cc:435] Acquired schema lock
0927 23:28:10.883619 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:10.883850 (+ 231us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:10.883883 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:10.883903 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:10.884035 (+ 132us) tablet.cc:699] Row locks acquired
0927 23:28:10.884047 (+ 12us) write_op.cc:260] PREPARE: finished
0927 23:28:10.884139 (+ 92us) write_op.cc:270] Start()
0927 23:28:10.884210 (+ 71us) write_op.cc:276] Timestamp: P: 1790551690884099 usec, L: 0
0927 23:28:10.884225 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:10.885144 (+ 919us) log.cc:842] Serialized 356 byte log entry
0927 23:28:10.966027 (+ 80883us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b58000a04b0. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb405d834 00007f8bb4062771 00007f8bb40626f9 00007f8bb4062689 00007f8bb4062651 00007f8bb406181d 0000000000580784 00000000005805b9 00007f8bad18e6b6 00007f8bad191b11
0927 23:28:11.027175 (+ 61148us) op_driver.cc:444] REPLICATION: finished
0927 23:28:11.027518 (+ 343us) write_op.cc:301] APPLY: starting
0927 23:28:11.027572 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:11.027668 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:11.027687 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:11.027861 (+ 174us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:11.028012 (+ 151us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:11.028057 (+ 45us) write_op.cc:312] APPLY: finished
0927 23:28:11.028229 (+ 172us) log.cc:842] Serialized 23 byte log entry
0927 23:28:11.032507 (+ 4278us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:11.032583 (+ 76us) write_op.cc:454] Released schema lock
0927 23:28:11.032616 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:11.032798 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:11.032821 (+ 23us) write_op.cc:454] Released schema lock
Related trace 'op':
0927 23:28:11.920229 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:11.955237 (+ 35008us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:11.955280 (+ 43us) write_op.cc:435] Acquired schema lock
0927 23:28:11.955293 (+ 13us) tablet.cc:660] Decoding operations
0927 23:28:11.955577 (+ 284us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:11.955611 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:11.955632 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:11.955769 (+ 137us) tablet.cc:699] Row locks acquired
0927 23:28:11.955782 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:28:11.955896 (+ 114us) write_op.cc:270] Start()
0927 23:28:11.955976 (+ 80us) write_op.cc:276] Timestamp: P: 1790551691952736 usec, L: 0
0927 23:28:11.955992 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:11.962692 (+ 6700us) log.cc:842] Serialized 356 byte log entry
0927 23:28:12.218137 (+255445us) op_driver.cc:444] REPLICATION: finished
0927 23:28:12.218808 (+ 671us) write_op.cc:301] APPLY: starting
0927 23:28:12.218867 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:12.218969 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:12.219059 (+ 90us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:12.219238 (+ 179us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:12.219405 (+ 167us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:12.219448 (+ 43us) write_op.cc:312] APPLY: finished
0927 23:28:12.220750 (+ 1302us) log.cc:842] Serialized 23 byte log entry
0927 23:28:12.221678 (+ 928us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:12.221741 (+ 63us) write_op.cc:454] Released schema lock
0927 23:28:12.221774 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:12.224054 (+ 2280us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:12.224090 (+ 36us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"spinlock_wait_cycles":181693312,"child_traces":[["op",{"apply.queue_time_us":181,"apply.run_cpu_time_us":1426,"apply.run_wall_time_us":5476,"num_ops":1,"prepare.queue_time_us":57748,"prepare.run_cpu_time_us":7493,"prepare.run_wall_time_us":75695,"raft.queue_time_us":57292,"raft.run_cpu_time_us":777,"raft.run_wall_time_us":23479,"replication_time_us":142876,"spinlock_wait_cycles":52273536,"thread_start_us":73504,"threads_started":1}],["op",{"apply.queue_time_us":510,"apply.run_cpu_time_us":1617,"apply.run_wall_time_us":5454,"num_ops":1,"prepare.queue_time_us":253712,"prepare.run_cpu_time_us":2095,"prepare.run_wall_time_us":43887,"replication_time_us":262064}]]}
I20260927 23:28:13.535028 3167 tablet_bootstrap.cc:654] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:13.567093 2932 tablet_service.cc:1514] Processing DeleteTablet for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae with delete_type TABLET_DATA_DELETED (replaced by tablet 8fadaafd49b04b9483a6a50f2f7d5be7) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:13.568725 2937 tablet_service.cc:1514] Processing DeleteTablet for tablet 4929fc9e396e48a189c94586d58aa55f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:05 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:13.569628 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet cc1c0da6ea5c4a09a71555f37becae33 with delete_type TABLET_DATA_DELETED (replaced by tablet 78e099028f7847d1b3de7d437be97de0) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:13.574823 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d0d8a82c5234880ab8396b1777e36c8" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:13.573740 3558 ts_tablet_manager.cc:1918] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:13.576494 3558 log.cc:1196] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/ac19099dd7bf4b669d504a654ee69482
I20260927 23:28:13.576871 2761 leader_election.cc:302] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:13.577397 3558 ts_tablet_manager.cc:1939] T ac19099dd7bf4b669d504a654ee69482 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:13.577596 3840 raft_consensus.cc:2726] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:13.585875 3072 raft_consensus.cc:348] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.587499 3072 raft_consensus.cc:374] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:13.587862 3072 raft_consensus.cc:729] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:13.589496 3072 consensus_queue.cc:260] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.602101 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 147) took 5184 ms. Trace:
I20260927 23:28:13.604238 3336 rpcz_store.cc:276] 0927 23:28:08.417897 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.422507 (+ 4610us) service_pool.cc:227] Handling call
0927 23:28:13.231578 (+4809071us) ts_tablet_manager.cc:1145] Deleting tablet 2411d7c53a154fafad9edda9fa75bb36
0927 23:28:13.231616 (+ 38us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.311914 (+ 80298us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.376690 (+ 64776us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.601861 (+225171us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":220095,"tablet-delete.queue_time_us":4808494}
I20260927 23:28:13.605952 3336 tablet_replica.cc:330] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:13.607254 3572 raft_consensus.cc:482] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:13.607869 3572 raft_consensus.cc:504] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.610309 3621 raft_consensus.cc:482] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:13.611020 3621 raft_consensus.cc:504] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.543072 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:13.612795 3559 tablet_replica.cc:330] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:13.613319 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 154) took 4201 ms. Trace:
I20260927 23:28:13.613523 3867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.60:41571 every 8 connection(s)
I20260927 23:28:13.613627 3559 raft_consensus.cc:2229] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:13.613644 3558 rpcz_store.cc:276] 0927 23:28:09.411877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.412036 (+ 159us) service_pool.cc:227] Handling call
0927 23:28:13.497512 (+4085476us) ts_tablet_manager.cc:1145] Deleting tablet ac19099dd7bf4b669d504a654ee69482
0927 23:28:13.497570 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.552772 (+ 55202us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.568226 (+ 15454us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.613240 (+ 45014us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4084876}
I20260927 23:28:13.614288 3298 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.60:41571
I20260927 23:28:13.614639 3559 raft_consensus.cc:2258] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:13.616707 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet ac19099dd7bf4b669d504a654ee69482 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]) successfully deleted
I20260927 23:28:13.617489 3559 ts_tablet_manager.cc:1905] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:13.617863 2390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_b5b37ba466074efda0f8867fc0af9e50"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:13.619356 2390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b5b37ba466074efda0f8867fc0af9e50 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:13.620460 3167 tablet_bootstrap.cc:492] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:13.620893 3167 ts_tablet_manager.cc:1397] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.123s user 0.008s sys 0.003s
I20260927 23:28:13.622601 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 2411d7c53a154fafad9edda9fa75bb36 (table default.Table_b967cf863a6848919c6542cb6bbb1c70 [id=597f1d374b754602850d1dbc3bd4e913]) successfully deleted
W20260927 23:28:13.623584 2379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 9eb27269a590476f878ee52a2aefd141 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 9eb27269a590476f878ee52a2aefd141 consensus state has no leader
I20260927 23:28:13.623281 3167 raft_consensus.cc:348] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:13.623725 2393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 46) took 862 ms (client timeout 1000 ms). Trace:
W20260927 23:28:13.624015 2393 rpcz_store.cc:269] 0927 23:28:12.760923 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.954149 (+193226us) service_pool.cc:227] Handling call
0927 23:28:13.042783 (+ 88634us) spinlock_profiling.cc:243] Waited 85.8 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:13.522111 (+479328us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b440009a790. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb1e0f301 00007f8bb1e2eb70 00007f8bb37bff1b 00007f8bb600491b 00007f8bb5fa9330 00007f8bb5faf083 00007f8bb60b24fe 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec
0927 23:28:13.623659 (+101548us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":10,"spinlock_wait_cycles":229665536}
I20260927 23:28:13.628165 3070 tablet_bootstrap.cc:492] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:13.628749 3070 ts_tablet_manager.cc:1397] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.141s user 0.011s sys 0.007s
I20260927 23:28:13.629544 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0115095a82489a8aeb941efcc8a979 with delete_type TABLET_DATA_DELETED (replaced by tablet 178503861b1648d9a931d1606d807ed8) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:13.630990 3070 raft_consensus.cc:348] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.631690 3070 raft_consensus.cc:374] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:13.631932 3070 raft_consensus.cc:729] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
W20260927 23:28:13.635102 2384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 32) took 1929 ms (client timeout 1000 ms). Trace:
I20260927 23:28:13.628001 2393 master_service.cc:946] ReplaceTablet: received request to replace tablet 0e0c12031a734ca0a6536404409c62ec from {username='slave'} at 127.0.0.1:51972
W20260927 23:28:13.635481 2384 rpcz_store.cc:269] 0927 23:28:11.705269 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:11.950006 (+244737us) service_pool.cc:227] Handling call
0927 23:28:12.013348 (+ 63342us) spinlock_profiling.cc:243] Waited 2.95 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:13.635004 (+1621656us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:13.056008 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:13.208977 (+152969us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:13.209006 (+ 29us) write_op.cc:435] Acquired schema lock
0927 23:28:13.209020 (+ 14us) tablet.cc:660] Decoding operations
0927 23:28:13.209859 (+ 839us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:13.209905 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:13.209927 (+ 22us) tablet.cc:683] Acquiring locks for 3 operations
0927 23:28:13.210169 (+ 242us) tablet.cc:699] Row locks acquired
0927 23:28:13.222122 (+ 11953us) write_op.cc:260] PREPARE: finished
0927 23:28:13.222271 (+ 149us) write_op.cc:270] Start()
0927 23:28:13.222369 (+ 98us) write_op.cc:276] Timestamp: P: 1790551693222181 usec, L: 0
0927 23:28:13.222387 (+ 18us) op_driver.cc:331] REPLICATION: starting
0927 23:28:13.222927 (+ 540us) log.cc:842] Serialized 1266 byte log entry
0927 23:28:13.237424 (+ 14497us) op_driver.cc:444] REPLICATION: finished
0927 23:28:13.237765 (+ 341us) write_op.cc:301] APPLY: starting
0927 23:28:13.237822 (+ 57us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:13.237946 (+ 124us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:13.238036 (+ 90us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:13.238358 (+ 322us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:13.238503 (+ 145us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0927 23:28:13.238558 (+ 55us) write_op.cc:312] APPLY: finished
0927 23:28:13.238797 (+ 239us) log.cc:842] Serialized 35 byte log entry
0927 23:28:13.238944 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.239019 (+ 75us) write_op.cc:454] Released schema lock
0927 23:28:13.239056 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:13.239215 (+ 159us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.239236 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":23881,"reported_tablets":16,"spinlock_wait_cycles":6777088,"child_traces":[["op",{"apply.queue_time_us":165,"apply.run_cpu_time_us":1652,"apply.run_wall_time_us":1651,"num_ops":3,"prepare.queue_time_us":479100,"prepare.run_cpu_time_us":2536,"prepare.run_wall_time_us":172398,"raft.queue_time_us":755,"raft.run_cpu_time_us":750,"raft.run_wall_time_us":42030,"replication_time_us":14953}]]}
I20260927 23:28:13.646603 3298 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/master-2/data/info.pb
I20260927 23:28:13.653542 3070 consensus_queue.cc:260] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.654772 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 3298
I20260927 23:28:13.643767 3558 tablet_replica.cc:330] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:13.657667 3558 raft_consensus.cc:2229] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:13.658334 3558 raft_consensus.cc:2258] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:13.661571 3558 ts_tablet_manager.cc:1905] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:13.623926 3167 raft_consensus.cc:374] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:13.665050 3167 raft_consensus.cc:729] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:13.672108 3167 consensus_queue.cc:260] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.677871 3070 ts_tablet_manager.cc:1428] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.049s user 0.006s sys 0.000s
I20260927 23:28:13.678879 3070 tablet_bootstrap.cc:492] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:13.684087 3796 raft_consensus.cc:482] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:13.684639 3796 raft_consensus.cc:504] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.687408 3796 leader_election.cc:288] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:13.689065 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eff8f3e5ef74e93b077b5e5bfb139c6" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:13.691007 3167 ts_tablet_manager.cc:1428] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.070s user 0.006s sys 0.000s
W20260927 23:28:13.690840 2609 leader_election.cc:341] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:13.713675 3167 tablet_bootstrap.cc:492] T 40bce5362d2541329b253716bbd99e07 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:13.722781 2387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_eaa0db537a9f4c38b1c0db99654a6292" } modify_external_catalogs: true
I20260927 23:28:13.723416 3572 leader_election.cc:288] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:13.728008 3621 leader_election.cc:288] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:13.736538 3559 ts_tablet_manager.cc:1918] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:13.742916 3559 log.cc:1196] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/5bf856a05dff4fb4943fe8e1d9b7f385
I20260927 23:28:13.748030 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eff8f3e5ef74e93b077b5e5bfb139c6" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:13.748778 2968 raft_consensus.cc:2454] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:13.763813 2384 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:13.760545 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40bce5362d2541329b253716bbd99e07" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:13.764775 2674 raft_consensus.cc:2454] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:13.761353 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eff8f3e5ef74e93b077b5e5bfb139c6" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:13.765918 2669 raft_consensus.cc:2454] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:13.767611 3559 ts_tablet_manager.cc:1939] T 5bf856a05dff4fb4943fe8e1d9b7f385 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:13.769441 3167 tablet_bootstrap.cc:654] T 40bce5362d2541329b253716bbd99e07 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:13.771332 3072 ts_tablet_manager.cc:1428] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.371s user 0.034s sys 0.025s
I20260927 23:28:13.772701 3072 tablet_bootstrap.cc:492] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:13.777127 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eff8f3e5ef74e93b077b5e5bfb139c6" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:13.782065 3559 tablet_replica.cc:330] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:13.789752 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40bce5362d2541329b253716bbd99e07" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:13.789834 3559 raft_consensus.cc:2229] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:13.792769 2608 leader_election.cc:302] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:13.793567 3748 raft_consensus.cc:2781] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:13.793889 3748 raft_consensus.cc:482] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:13.794157 3748 raft_consensus.cc:3037] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:13.796223 3621 consensus_queue.cc:1035] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:28:13.800254 2906 leader_election.cc:341] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:13.800942 2906 leader_election.cc:341] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:13.802485 3748 raft_consensus.cc:504] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.806133 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eff8f3e5ef74e93b077b5e5bfb139c6" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:13.805613 3559 raft_consensus.cc:2258] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:13.808182 2609 leader_election.cc:341] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:13.810142 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eff8f3e5ef74e93b077b5e5bfb139c6" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:13.810765 2968 raft_consensus.cc:3037] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:13.814219 2907 leader_election.cc:302] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:13.820470 3336 raft_consensus.cc:2229] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:13.821277 3336 raft_consensus.cc:2258] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:13.822801 3070 tablet_bootstrap.cc:654] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:13.820259 3559 ts_tablet_manager.cc:1905] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:13.824028 2968 raft_consensus.cc:2454] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:13.826007 3336 ts_tablet_manager.cc:1905] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:13.828764 3796 raft_consensus.cc:482] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:13.829236 3796 raft_consensus.cc:504] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.829970 3167 tablet_bootstrap.cc:492] T 40bce5362d2541329b253716bbd99e07 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:13.830402 3167 ts_tablet_manager.cc:1397] T 40bce5362d2541329b253716bbd99e07 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.117s user 0.010s sys 0.000s
I20260927 23:28:13.834030 3167 raft_consensus.cc:348] T 40bce5362d2541329b253716bbd99e07 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.837623 3796 leader_election.cc:288] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:13.836290 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "371842420a6048fca3cbc02d18f74ffc" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:13.838235 3167 raft_consensus.cc:374] T 40bce5362d2541329b253716bbd99e07 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:13.838532 3167 raft_consensus.cc:729] T 40bce5362d2541329b253716bbd99e07 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
W20260927 23:28:13.839246 2609 leader_election.cc:341] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:13.839129 3167 consensus_queue.cc:260] T 40bce5362d2541329b253716bbd99e07 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.824281 3748 leader_election.cc:288] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:13.840561 2608 leader_election.cc:302] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:13.841223 3796 raft_consensus.cc:2781] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:13.842423 3796 raft_consensus.cc:686] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:13.845492 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "371842420a6048fca3cbc02d18f74ffc" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:13.844532 3796 consensus_queue.cc:237] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.827168 3558 ts_tablet_manager.cc:1918] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:13.853631 3558 log.cc:1196] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/fd0115095a82489a8aeb941efcc8a979
I20260927 23:28:13.854640 3558 ts_tablet_manager.cc:1939] T fd0115095a82489a8aeb941efcc8a979 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:13.860229 3572 raft_consensus.cc:2781] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:13.860694 3572 raft_consensus.cc:482] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:13.861008 3572 raft_consensus.cc:3037] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:13.870441 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 157) took 4295 ms. Trace:
I20260927 23:28:13.871078 3558 rpcz_store.cc:276] 0927 23:28:09.574726 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.574924 (+ 198us) service_pool.cc:227] Handling call
0927 23:28:13.643583 (+4068659us) ts_tablet_manager.cc:1145] Deleting tablet fd0115095a82489a8aeb941efcc8a979
0927 23:28:13.643636 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.756973 (+113337us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.826926 (+ 69953us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:13.870352 (+ 43426us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4068037}
I20260927 23:28:13.872923 3572 raft_consensus.cc:504] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.888271 3572 leader_election.cc:288] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:13.889000 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 74981us
I20260927 23:28:13.889933 2907 leader_election.cc:302] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:13.891301 2385 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1625e523e1ca40b6b8324f302bb70d31 consensus state has no leader
I20260927 23:28:13.891844 2385 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:51972 (request call id 276) took 1049 ms. Trace:
I20260927 23:28:13.892865 2386 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 274) took 1245 ms. Trace:
I20260927 23:28:13.892215 2385 rpcz_store.cc:276] 0927 23:28:12.842438 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.521775 (+679337us) service_pool.cc:227] Handling call
0927 23:28:13.722374 (+200599us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:13.745590 (+ 23216us) catalog_manager.cc:3616] Apply alter schema
0927 23:28:13.746178 (+ 588us) catalog_manager.cc:3639] Acquired catalog manager lock
0927 23:28:13.746374 (+ 196us) catalog_manager.cc:3786] Updating metadata on disk
0927 23:28:13.890666 (+144292us) catalog_manager.cc:3817] Committing alterations to in-memory state
0927 23:28:13.891789 (+ 1123us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:13.768226 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:13.816796 (+ 48570us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:13.816824 (+ 28us) write_op.cc:435] Acquired schema lock
0927 23:28:13.816836 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:13.817128 (+ 292us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:13.817171 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:13.817191 (+ 20us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:13.817353 (+ 162us) tablet.cc:699] Row locks acquired
0927 23:28:13.817367 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:13.817491 (+ 124us) write_op.cc:270] Start()
0927 23:28:13.817584 (+ 93us) write_op.cc:276] Timestamp: P: 1790551693817447 usec, L: 0
0927 23:28:13.817600 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:13.818129 (+ 529us) log.cc:842] Serialized 405 byte log entry
0927 23:28:13.888695 (+ 70566us) op_driver.cc:444] REPLICATION: finished
0927 23:28:13.889437 (+ 742us) write_op.cc:301] APPLY: starting
0927 23:28:13.889495 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:13.889583 (+ 88us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:13.889669 (+ 86us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:13.889876 (+ 207us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:13.890003 (+ 127us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:13.890044 (+ 41us) write_op.cc:312] APPLY: finished
0927 23:28:13.890212 (+ 168us) log.cc:842] Serialized 23 byte log entry
0927 23:28:13.890340 (+ 128us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.890412 (+ 72us) write_op.cc:454] Released schema lock
0927 23:28:13.890443 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:13.890630 (+ 187us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.890654 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":590,"apply.run_cpu_time_us":1374,"apply.run_wall_time_us":1373,"num_ops":1,"prepare.queue_time_us":20689,"prepare.run_cpu_time_us":2210,"prepare.run_wall_time_us":50229,"raft.queue_time_us":41391,"raft.run_cpu_time_us":1084,"raft.run_wall_time_us":18104,"replication_time_us":71021}]]}
I20260927 23:28:13.893267 2386 rpcz_store.cc:276] 0927 23:28:12.647492 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.349759 (+702267us) service_pool.cc:227] Handling call
0927 23:28:13.434301 (+ 84542us) spinlock_profiling.cc:243] Waited 83.7 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b3156 00007f8bb3bf0034 00007f8bb3beffc2 00007f8bb3befef1 00007f8bb3befe6c 00007f8bb3bef122 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:13.716428 (+282127us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:13.719445 (+ 3017us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:13.892232 (+172787us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:13.892569 (+ 337us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:13.892811 (+ 242us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:13.818558 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:13.859908 (+ 41350us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:13.859935 (+ 27us) write_op.cc:435] Acquired schema lock
0927 23:28:13.859946 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:13.860232 (+ 286us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:13.860274 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:13.860321 (+ 47us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:13.860537 (+ 216us) tablet.cc:699] Row locks acquired
0927 23:28:13.860551 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:13.860682 (+ 131us) write_op.cc:270] Start()
0927 23:28:13.860787 (+ 105us) write_op.cc:276] Timestamp: P: 1790551693860632 usec, L: 0
0927 23:28:13.860804 (+ 17us) op_driver.cc:331] REPLICATION: starting
0927 23:28:13.862683 (+ 1879us) log.cc:842] Serialized 433 byte log entry
0927 23:28:13.884048 (+ 21365us) spinlock_profiling.cc:243] Waited 7.4 ms on lock 0x7b58000a01b0. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb405d834 00007f8bb4062771 00007f8bb40626f9 00007f8bb4062689 00007f8bb4062651 00007f8bb406181d 0000000000580784 00000000005805b9 00007f8bad18e6b6 00007f8bad191b11
0927 23:28:13.889026 (+ 4978us) op_driver.cc:444] REPLICATION: finished
0927 23:28:13.890927 (+ 1901us) write_op.cc:301] APPLY: starting
0927 23:28:13.890976 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:13.891074 (+ 98us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:13.891092 (+ 18us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:13.891454 (+ 362us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:13.891588 (+ 134us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:13.891632 (+ 44us) write_op.cc:312] APPLY: finished
0927 23:28:13.891806 (+ 174us) log.cc:842] Serialized 29 byte log entry
0927 23:28:13.891942 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.892003 (+ 61us) write_op.cc:454] Released schema lock
0927 23:28:13.892032 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:13.892195 (+ 163us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.892217 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":192421760,"child_traces":[["op",{"apply.queue_time_us":1790,"apply.run_cpu_time_us":1471,"apply.run_wall_time_us":2311,"num_ops":2,"prepare.queue_time_us":67520,"prepare.run_cpu_time_us":2227,"prepare.run_wall_time_us":44407,"raft.queue_time_us":1588,"raft.run_cpu_time_us":1480,"raft.run_wall_time_us":10051,"replication_time_us":28154,"spinlock_wait_cycles":17016704,"threads_started":1}]]}
I20260927 23:28:13.900437 3072 tablet_bootstrap.cc:654] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:13.773242 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 5bf856a05dff4fb4943fe8e1d9b7f385 (table default.Table_25f0e3650873407a8b3cbd8c8e0c86af [id=48f252a9cdb34eb1b452540f4186285c]) successfully deleted
W20260927 23:28:13.913293 2380 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 140072us
W20260927 23:28:13.915426 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 67627.6us
W20260927 23:28:13.916500 2608 leader_election.cc:341] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:13.917024 2608 leader_election.cc:302] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:13.917780 3748 raft_consensus.cc:2726] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:13.919498 3070 tablet_bootstrap.cc:492] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:13.920014 3070 ts_tablet_manager.cc:1397] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.241s user 0.010s sys 0.001s
I20260927 23:28:13.922207 3866 raft_consensus.cc:2741] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260927 23:28:13.865741 3167 ts_tablet_manager.cc:1428] T 40bce5362d2541329b253716bbd99e07 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.035s user 0.006s sys 0.000s
I20260927 23:28:13.924976 3070 raft_consensus.cc:348] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.925691 3070 raft_consensus.cc:374] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:13.925952 3070 raft_consensus.cc:729] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:13.931351 3070 consensus_queue.cc:260] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.934238 3167 tablet_bootstrap.cc:492] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
W20260927 23:28:13.934859 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 45008.3us
I20260927 23:28:13.937817 3070 ts_tablet_manager.cc:1428] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.017s user 0.005s sys 0.001s
I20260927 23:28:13.940335 3070 tablet_bootstrap.cc:492] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:13.946398 3866 raft_consensus.cc:482] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5ce671f028a34ea4b8f0520c2689c9a1)
I20260927 23:28:13.947036 3866 raft_consensus.cc:504] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.949234 3866 leader_election.cc:288] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:13.964063 2907 leader_election.cc:341] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: cfbbe17f54d1418b95efc3943db0b6c1
I20260927 23:28:13.965039 3866 consensus_queue.cc:1035] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:13.945363 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40bce5362d2541329b253716bbd99e07" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:13.977286 3017 heartbeater.cc:347] Connected to a master server at 127.2.57.60:41571
I20260927 23:28:13.977340 2669 raft_consensus.cc:3037] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:13.975241 2721 heartbeater.cc:347] Connected to a master server at 127.2.57.60:41571
I20260927 23:28:13.985821 3072 tablet_bootstrap.cc:492] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:13.985760 2669 raft_consensus.cc:2454] T 40bce5362d2541329b253716bbd99e07 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:13.987721 3072 ts_tablet_manager.cc:1397] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.215s user 0.010s sys 0.002s
I20260927 23:28:13.990792 3072 raft_consensus.cc:348] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:13.991802 3871 sys_catalog.cc:263] Verifying existing consensus state
I20260927 23:28:13.975241 3559 ts_tablet_manager.cc:1918] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:13.998471 3559 log.cc:1196] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/fd0115095a82489a8aeb941efcc8a979
I20260927 23:28:13.999725 3559 ts_tablet_manager.cc:1939] T fd0115095a82489a8aeb941efcc8a979 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:14.014102 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 158) took 4416 ms. Trace:
I20260927 23:28:14.016470 3558 rpcz_store.cc:276] 0927 23:28:09.597505 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.597643 (+ 138us) service_pool.cc:227] Handling call
0927 23:28:13.925003 (+4327360us) ts_tablet_manager.cc:1145] Deleting tablet 8b891bab5fb343b0bb94641aea7662f0
0927 23:28:13.935975 (+ 10972us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.014024 (+ 78049us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4326718}
I20260927 23:28:14.003767 3070 tablet_bootstrap.cc:654] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:14.032781 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40bce5362d2541329b253716bbd99e07" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:14.033488 2820 raft_consensus.cc:3037] T 40bce5362d2541329b253716bbd99e07 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:14.034201 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 159) took 4405 ms. Trace:
I20260927 23:28:14.034639 3558 rpcz_store.cc:276] 0927 23:28:09.629184 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.629320 (+ 136us) service_pool.cc:227] Handling call
0927 23:28:14.017896 (+4388576us) ts_tablet_manager.cc:1145] Deleting tablet a1890852164d492db3540a97847888fd
0927 23:28:14.033840 (+ 15944us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.034144 (+ 304us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4387928}
I20260927 23:28:14.036060 3072 raft_consensus.cc:374] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:14.036404 3072 raft_consensus.cc:729] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:14.036259 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 178) took 3652 ms. Trace:
I20260927 23:28:14.040567 3558 rpcz_store.cc:276] 0927 23:28:10.383717 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.385777 (+ 2060us) service_pool.cc:227] Handling call
0927 23:28:14.035864 (+3650087us) ts_tablet_manager.cc:1145] Deleting tablet ee5d073c8df84f0884dc0f0d88eeed9f
0927 23:28:14.035919 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.036198 (+ 279us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3649413}
I20260927 23:28:14.041726 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet fd0115095a82489a8aeb941efcc8a979 (table default.Table_0df33121d2eb4445ac8b8c6a9ee44d24 [id=0a5810627a964e7caf66c69e7c3ec9b4]) successfully deleted
W20260927 23:28:14.042874 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 8b891bab5fb343b0bb94641aea7662f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b891bab5fb343b0bb94641aea7662f0 already in progress: creating tablet
W20260927 23:28:14.043638 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet a1890852164d492db3540a97847888fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1890852164d492db3540a97847888fd already in progress: creating tablet
W20260927 23:28:14.044613 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet ee5d073c8df84f0884dc0f0d88eeed9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee5d073c8df84f0884dc0f0d88eeed9f already in progress: creating tablet
I20260927 23:28:14.046483 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet fd0115095a82489a8aeb941efcc8a979 (table default.Table_0df33121d2eb4445ac8b8c6a9ee44d24 [id=0a5810627a964e7caf66c69e7c3ec9b4]) successfully deleted
I20260927 23:28:14.037102 3072 consensus_queue.cc:260] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.049701 2384 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1da38499af164d7fb5c1a8d65167ad24 (table default.Table_f7b5c90ef9a94341a88b0a40ea0f7e1e [id=b12aae65f933476ab57974d4b328a266]). Expected version 3 got 0
I20260927 23:28:14.050239 3072 ts_tablet_manager.cc:1428] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.062s user 0.005s sys 0.000s
I20260927 23:28:14.050956 2384 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:14.051268 3072 tablet_bootstrap.cc:492] T a708a517b744496c83a3aa759b3f1081 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:14.057897 2386 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_f7b5c90ef9a94341a88b0a40ea0f7e1e" } modify_external_catalogs: true
I20260927 23:28:14.058863 2386 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_f7b5c90ef9a94341a88b0a40ea0f7e1e" } modify_external_catalogs: true
I20260927 23:28:14.064909 2385 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_58948ac6bf854c31a26e8a6d47456c90" } new_table_name: "default.Table_55dca2c970f045b6966759363c4a0816" modify_external_catalogs: true
I20260927 23:28:14.068837 2388 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_48ee3ed6e26d4e41aa46aaf16ff3a09d [id=21118fc25eb9485b92a9034bbeef8b13] alter complete (version 1)
I20260927 23:28:14.070748 2873 heartbeater.cc:347] Connected to a master server at 127.2.57.60:41571
I20260927 23:28:14.073529 3336 ts_tablet_manager.cc:1918] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:14.073721 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 182) took 3600 ms. Trace:
I20260927 23:28:14.074018 3336 log.cc:1196] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/5bf856a05dff4fb4943fe8e1d9b7f385
I20260927 23:28:14.074064 3558 rpcz_store.cc:276] 0927 23:28:10.472994 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.505982 (+ 32988us) service_pool.cc:227] Handling call
0927 23:28:14.073256 (+3567274us) ts_tablet_manager.cc:1145] Deleting tablet bbeeccc19a504e7984b293127766daf0
0927 23:28:14.073314 (+ 58us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.073645 (+ 331us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3556524}
I20260927 23:28:14.075007 3336 ts_tablet_manager.cc:1939] T 5bf856a05dff4fb4943fe8e1d9b7f385 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:14.075181 3558 tablet_replica.cc:330] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:14.075898 3558 raft_consensus.cc:2229] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:14.076143 3070 tablet_bootstrap.cc:492] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:14.076581 3070 ts_tablet_manager.cc:1397] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.136s user 0.014s sys 0.000s
I20260927 23:28:14.081387 3558 raft_consensus.cc:2258] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:14.083421 3558 ts_tablet_manager.cc:1905] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:14.084759 2906 leader_election.cc:341] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Not found: Tablet not found: cfbbe17f54d1418b95efc3943db0b6c1
I20260927 23:28:14.085217 2906 leader_election.cc:302] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:14.087225 2389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 29) took 1064 ms (client timeout 1000 ms). Trace:
I20260927 23:28:14.087966 3070 raft_consensus.cc:348] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.088708 3070 raft_consensus.cc:374] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:14.088869 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet 2411d7c53a154fafad9edda9fa75bb36 with delete_type TABLET_DATA_DELETED (replaced by tablet 73cfb41befe44e5189029abafe7e326d) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:14.088954 3070 raft_consensus.cc:729] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:14.089567 3866 raft_consensus.cc:2726] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:14.089545 3070 consensus_queue.cc:260] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.100842 3871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: Bootstrap starting.
I20260927 23:28:14.110049 2392 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 275) took 1449 ms. Trace:
I20260927 23:28:14.093165 3072 tablet_bootstrap.cc:654] T a708a517b744496c83a3aa759b3f1081 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:14.112100 3070 ts_tablet_manager.cc:1428] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.035s user 0.005s sys 0.000s
I20260927 23:28:14.113126 3070 tablet_bootstrap.cc:492] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:14.110442 2392 rpcz_store.cc:276] 0927 23:28:12.660367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.378495 (+718128us) service_pool.cc:227] Handling call
0927 23:28:13.435994 (+ 57499us) spinlock_profiling.cc:243] Waited 56.5 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b3156 00007f8bb3bf0034 00007f8bb3beffc2 00007f8bb3befef1 00007f8bb3befe6c 00007f8bb3bef122 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:13.722601 (+286607us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:13.755959 (+ 33358us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:14.092041 (+336082us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:14.109427 (+ 17386us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:14.109866 (+ 439us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:13.885677 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:13.886038 (+ 361us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:13.886059 (+ 21us) write_op.cc:435] Acquired schema lock
0927 23:28:13.886073 (+ 14us) tablet.cc:660] Decoding operations
0927 23:28:13.886326 (+ 253us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:13.886360 (+ 34us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:13.886382 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:13.886564 (+ 182us) tablet.cc:699] Row locks acquired
0927 23:28:13.886578 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:13.886679 (+ 101us) write_op.cc:270] Start()
0927 23:28:13.886754 (+ 75us) write_op.cc:276] Timestamp: P: 1790551693886633 usec, L: 0
0927 23:28:13.886769 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:13.887251 (+ 482us) log.cc:842] Serialized 433 byte log entry
0927 23:28:13.969035 (+ 81784us) op_driver.cc:444] REPLICATION: finished
0927 23:28:13.970954 (+ 1919us) write_op.cc:301] APPLY: starting
0927 23:28:13.971002 (+ 48us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:13.971098 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:13.971176 (+ 78us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:13.971538 (+ 362us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:13.971670 (+ 132us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:13.971710 (+ 40us) write_op.cc:312] APPLY: finished
0927 23:28:13.971879 (+ 169us) log.cc:842] Serialized 29 byte log entry
0927 23:28:13.971997 (+ 118us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.972051 (+ 54us) write_op.cc:454] Released schema lock
0927 23:28:13.972083 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:13.972233 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.972258 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":129959680,"child_traces":[["op",{"apply.queue_time_us":1809,"apply.run_cpu_time_us":1481,"apply.run_wall_time_us":2534,"num_ops":2,"prepare.queue_time_us":126253,"prepare.run_cpu_time_us":1735,"prepare.run_wall_time_us":1733,"replication_time_us":82200}]]}
I20260927 23:28:14.116133 3840 raft_consensus.cc:482] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:14.116652 3840 raft_consensus.cc:504] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.116452 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_1e1f242e1c804c9b883131486cbc0125"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:14.118382 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e1f242e1c804c9b883131486cbc0125 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:14.118635 3840 leader_election.cc:288] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:14.122064 2761 leader_election.cc:341] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: 2411d7c53a154fafad9edda9fa75bb36
I20260927 23:28:14.129566 2387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc
I20260927 23:28:14.131235 2387 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 277) took 1282 ms. Trace:
I20260927 23:28:14.133481 2393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 0e0c12031a734ca0a6536404409c62ec
I20260927 23:28:14.134680 3070 tablet_bootstrap.cc:654] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:14.134864 2393 catalog_manager.cc:6565] ReplaceTablet: tablet 0e0c12031a734ca0a6536404409c62ec deleted and replaced by tablet 2c99ea48bb3e461e9fc0f4d95052c663
I20260927 23:28:14.135733 2393 master_service.cc:946] ReplaceTablet: received request to replace tablet 3a8dbc1165ac4f5b8ed792f6064cce6f from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:14.141892 2651 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0c12031a734ca0a6536404409c62ec with delete_type TABLET_DATA_DELETED (replaced by tablet 2c99ea48bb3e461e9fc0f4d95052c663) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:14.141911 2649 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:13 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:14.144747 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 0e0c12031a734ca0a6536404409c62ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e0c12031a734ca0a6536404409c62ec already in progress: creating tablet
W20260927 23:28:14.145401 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc already in progress: creating tablet
I20260927 23:28:14.131580 2387 rpcz_store.cc:276] 0927 23:28:12.848464 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.524018 (+675554us) service_pool.cc:227] Handling call
0927 23:28:13.723537 (+199519us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:13.753729 (+ 30192us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:28:13.753888 (+ 159us) catalog_manager.cc:2761] Locking tablets
0927 23:28:13.754635 (+ 747us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:28:14.128728 (+374093us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:28:14.129145 (+ 417us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:14.129324 (+ 179us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:14.131143 (+ 1819us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:13.965318 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:13.965704 (+ 386us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:13.965740 (+ 36us) write_op.cc:435] Acquired schema lock
0927 23:28:13.965754 (+ 14us) tablet.cc:660] Decoding operations
0927 23:28:13.966071 (+ 317us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:13.966109 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:13.966130 (+ 21us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:13.966401 (+ 271us) tablet.cc:699] Row locks acquired
0927 23:28:13.966415 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:13.966521 (+ 106us) write_op.cc:270] Start()
0927 23:28:13.966607 (+ 86us) write_op.cc:276] Timestamp: P: 1790551693966466 usec, L: 0
0927 23:28:13.966624 (+ 17us) op_driver.cc:331] REPLICATION: starting
0927 23:28:13.967208 (+ 584us) log.cc:842] Serialized 753 byte log entry
0927 23:28:14.124308 (+157100us) op_driver.cc:444] REPLICATION: finished
0927 23:28:14.125294 (+ 986us) write_op.cc:301] APPLY: starting
0927 23:28:14.125369 (+ 75us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:14.125511 (+ 142us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:14.125645 (+ 134us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:14.126017 (+ 372us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:14.126162 (+ 145us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:14.126226 (+ 64us) write_op.cc:312] APPLY: finished
0927 23:28:14.126407 (+ 181us) log.cc:842] Serialized 29 byte log entry
0927 23:28:14.126625 (+ 218us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:14.126720 (+ 95us) write_op.cc:454] Released schema lock
0927 23:28:14.126752 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:14.126998 (+ 246us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:14.127021 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":725,"apply.run_cpu_time_us":1960,"apply.run_wall_time_us":1960,"mutex_wait_us":14773,"num_ops":2,"prepare.queue_time_us":100201,"prepare.run_cpu_time_us":2024,"prepare.run_wall_time_us":3212,"replication_time_us":157554,"spinlock_wait_cycles":179328}]]}
I20260927 23:28:14.159389 2798 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:13 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:14.160109 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0c12031a734ca0a6536404409c62ec with delete_type TABLET_DATA_DELETED (replaced by tablet 2c99ea48bb3e461e9fc0f4d95052c663) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:14.161769 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2411d7c53a154fafad9edda9fa75bb36" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:14.162063 2826 raft_consensus.cc:3037] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:14.162453 2672 raft_consensus.cc:2454] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:28:14.163848 2763 leader_election.cc:302] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:14.165365 3840 raft_consensus.cc:2781] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:14.165746 3840 raft_consensus.cc:482] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:14.166117 3840 raft_consensus.cc:3037] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:14.173840 3871 log.cc:824] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: Log is configured to *not* fsync() on all Append() calls
W20260927 23:28:14.184468 2389 rpcz_store.cc:269] 0927 23:28:13.022365 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.124018 (+101653us) service_pool.cc:227] Handling call
0927 23:28:14.087019 (+963001us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:13.617686 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:13.618070 (+ 384us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:13.618095 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:28:13.618107 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:13.618438 (+ 331us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:13.618481 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:13.618503 (+ 22us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:13.618792 (+ 289us) tablet.cc:699] Row locks acquired
0927 23:28:13.618806 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:13.618918 (+ 112us) write_op.cc:270] Start()
0927 23:28:13.618999 (+ 81us) write_op.cc:276] Timestamp: P: 1790551693618850 usec, L: 0
0927 23:28:13.619015 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:13.619542 (+ 527us) log.cc:842] Serialized 895 byte log entry
0927 23:28:13.665315 (+ 45773us) op_driver.cc:444] REPLICATION: finished
0927 23:28:13.666528 (+ 1213us) write_op.cc:301] APPLY: starting
0927 23:28:13.666596 (+ 68us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:13.666731 (+ 135us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:13.666849 (+ 118us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:13.667135 (+ 286us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:13.667300 (+ 165us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:13.667345 (+ 45us) write_op.cc:312] APPLY: finished
0927 23:28:13.667576 (+ 231us) log.cc:842] Serialized 29 byte log entry
0927 23:28:13.668353 (+ 777us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.668437 (+ 84us) write_op.cc:454] Released schema lock
0927 23:28:13.668483 (+ 46us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:13.669795 (+ 1312us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.669823 (+ 28us) write_op.cc:454] Released schema lock
0927 23:28:13.683510 (+ 13687us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b180005e960. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb5fd48b1 00007f8bb5fe1661 00007f8bad18e764 00007f8bad191b11 00007f8bad191ac9 00007f8bad191a59 00007f8bad191a21 00007f8bad190d1d 0000000000580784 00000000005805b9 00007f8bad170dc6
Metrics: {"mutex_wait_us":5064,"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":1004,"apply.run_cpu_time_us":1902,"apply.run_wall_time_us":3540,"num_ops":2,"prepare.queue_time_us":159,"prepare.run_cpu_time_us":2165,"prepare.run_wall_time_us":3898,"raft.queue_time_us":84,"raft.run_cpu_time_us":1166,"raft.run_wall_time_us":31665,"replication_time_us":46216,"spinlock_wait_cycles":13507328}]]}
I20260927 23:28:14.188038 3840 raft_consensus.cc:504] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.190177 2820 raft_consensus.cc:2454] T 40bce5362d2541329b253716bbd99e07 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:14.191538 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2411d7c53a154fafad9edda9fa75bb36" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:14.192138 2672 raft_consensus.cc:3037] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:14.197392 3167 tablet_bootstrap.cc:654] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:14.199518 2672 raft_consensus.cc:2454] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:28:14.200670 2763 leader_election.cc:302] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:14.216746 3840 leader_election.cc:288] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:14.218731 2761 leader_election.cc:341] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: 2411d7c53a154fafad9edda9fa75bb36
I20260927 23:28:14.220685 3840 raft_consensus.cc:2781] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:14.223030 3840 raft_consensus.cc:686] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Becoming Leader. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Running, Role: LEADER
I20260927 23:28:14.226140 3840 consensus_queue.cc:237] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.228612 3070 tablet_bootstrap.cc:492] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:14.229046 3070 ts_tablet_manager.cc:1397] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.116s user 0.013s sys 0.021s
I20260927 23:28:13.993690 2907 leader_election.cc:302] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:14.147953 3072 tablet_bootstrap.cc:492] T a708a517b744496c83a3aa759b3f1081 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:14.159130 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:13 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:14.160334 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0c12031a734ca0a6536404409c62ec with delete_type TABLET_DATA_DELETED (replaced by tablet 2c99ea48bb3e461e9fc0f4d95052c663) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:14.171447 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 153) took 5693 ms. Trace:
I20260927 23:28:14.232880 3072 ts_tablet_manager.cc:1397] T a708a517b744496c83a3aa759b3f1081 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.182s user 0.011s sys 0.000s
I20260927 23:28:14.233050 3336 rpcz_store.cc:276] 0927 23:28:08.478491 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.478594 (+ 103us) service_pool.cc:227] Handling call
0927 23:28:13.605686 (+5127092us) ts_tablet_manager.cc:1145] Deleting tablet 5bf856a05dff4fb4943fe8e1d9b7f385
0927 23:28:13.605771 (+ 85us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:13.907950 (+302179us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:14.073249 (+165299us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:14.171377 (+ 98128us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":212989,"tablet-delete.queue_time_us":5126420}
I20260927 23:28:14.234437 3336 tablet_replica.cc:330] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:14.235224 3336 raft_consensus.cc:2229] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:14.235706 3336 raft_consensus.cc:2258] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:14.235436 3072 raft_consensus.cc:348] T a708a517b744496c83a3aa759b3f1081 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.236701 3748 raft_consensus.cc:482] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:14.237146 3748 raft_consensus.cc:504] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.238749 3336 ts_tablet_manager.cc:1905] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:14.239162 3748 leader_election.cc:288] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:14.236126 3072 raft_consensus.cc:374] T a708a517b744496c83a3aa759b3f1081 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:14.240639 3072 raft_consensus.cc:729] T a708a517b744496c83a3aa759b3f1081 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:14.241163 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0578b6c99c4209a8d3c76f2089ef46" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:14.241535 3072 consensus_queue.cc:260] T a708a517b744496c83a3aa759b3f1081 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:14.242429 2609 leader_election.cc:341] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:14.243799 3072 ts_tablet_manager.cc:1428] T a708a517b744496c83a3aa759b3f1081 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
W20260927 23:28:14.244797 2763 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 44224.3us
W20260927 23:28:14.245218 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 51118.3us
I20260927 23:28:14.246362 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0578b6c99c4209a8d3c76f2089ef46" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:14.246877 3072 tablet_bootstrap.cc:492] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
W20260927 23:28:14.248027 2608 leader_election.cc:341] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:14.248463 2608 leader_election.cc:302] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:14.231359 3070 raft_consensus.cc:348] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.249411 3748 raft_consensus.cc:2726] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:14.251231 3070 raft_consensus.cc:374] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:14.251518 3070 raft_consensus.cc:729] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:14.252240 3167 tablet_bootstrap.cc:492] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:14.252178 3070 consensus_queue.cc:260] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.252697 3167 ts_tablet_manager.cc:1397] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.319s user 0.006s sys 0.006s
I20260927 23:28:14.260972 3070 ts_tablet_manager.cc:1428] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.032s user 0.006s sys 0.000s
I20260927 23:28:14.261958 3070 tablet_bootstrap.cc:492] T 5c87d497da884544920b3be9c78daccf P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:14.266301 3167 raft_consensus.cc:348] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.275674 3072 tablet_bootstrap.cc:654] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:14.276780 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 283176us
I20260927 23:28:14.280535 3871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 23:28:14.281643 3871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: Bootstrap complete.
I20260927 23:28:14.287016 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 5bf856a05dff4fb4943fe8e1d9b7f385 (table default.Table_25f0e3650873407a8b3cbd8c8e0c86af [id=48f252a9cdb34eb1b452540f4186285c]) successfully deleted
I20260927 23:28:14.267041 3167 raft_consensus.cc:374] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:14.296685 3167 raft_consensus.cc:729] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:14.297502 3167 consensus_queue.cc:260] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.299578 3167 ts_tablet_manager.cc:1428] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.047s user 0.004s sys 0.001s
I20260927 23:28:14.305459 2387 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_8a68a0857d4f40a99f630c8ac395c419" } new_table_name: "default.Table_0c79af57138b40d28c6a5e46e03d3c2e" modify_external_catalogs: true
I20260927 23:28:14.312032 3748 raft_consensus.cc:482] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:14.312568 3748 raft_consensus.cc:504] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.314538 3748 leader_election.cc:288] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:14.318037 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a708a517b744496c83a3aa759b3f1081" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:14.318421 3866 raft_consensus.cc:2781] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:14.318706 2968 raft_consensus.cc:2454] T a708a517b744496c83a3aa759b3f1081 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:14.318904 3866 raft_consensus.cc:686] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:14.319872 2608 leader_election.cc:302] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:14.320696 3748 raft_consensus.cc:2781] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:14.321012 3748 raft_consensus.cc:482] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:14.321305 3748 raft_consensus.cc:3037] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:14.323944 3871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:28:14.328076 3871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cdd64d48a684d5ebccc2e2e8a4e3150, State: Initialized, Role: FOLLOWER
I20260927 23:28:14.329337 3871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:28:14.329530 3748 raft_consensus.cc:504] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.338306 3902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:28:14.339311 3902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:28:14.341455 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_341a1994d8db4935ae73da15c9261fc7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:14.342446 3871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:28:14.342449 3748 leader_election.cc:288] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:14.343071 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_341a1994d8db4935ae73da15c9261fc7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:14.319799 3866 consensus_queue.cc:237] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.373905 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a708a517b744496c83a3aa759b3f1081" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:14.374598 2968 raft_consensus.cc:3037] T a708a517b744496c83a3aa759b3f1081 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:14.375475 3336 ts_tablet_manager.cc:1918] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:14.375865 3336 log.cc:1196] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/3f71f948891b4d7fa95c3a1fbd92ce1e
I20260927 23:28:14.376561 3336 ts_tablet_manager.cc:1939] T 3f71f948891b4d7fa95c3a1fbd92ce1e P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:14.300585 3167 tablet_bootstrap.cc:492] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:14.360435 3840 raft_consensus.cc:482] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:14.316654 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a708a517b744496c83a3aa759b3f1081" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:14.384773 2609 leader_election.cc:341] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:14.332828 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a708a517b744496c83a3aa759b3f1081" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:14.386669 3558 ts_tablet_manager.cc:1918] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:14.357504 3897 raft_consensus.cc:482] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:14.387199 3897 raft_consensus.cc:504] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.391453 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eff8f3e5ef74e93b077b5e5bfb139c6" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:14.395682 2609 leader_election.cc:341] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:14.396833 3748 raft_consensus.cc:482] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:14.382205 3840 raft_consensus.cc:504] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.397270 3748 raft_consensus.cc:504] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.399060 3748 leader_election.cc:288] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:14.399063 3840 leader_election.cc:288] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:14.399298 2387 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4322d56dd17347d0bcad0127f3869ae2 consensus state has no leader
I20260927 23:28:14.399497 3897 leader_election.cc:288] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:14.400738 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_0fd5cd6b3a5c4d0598149d887222ad33"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:14.401309 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56766ed12a7c4a668a7e08fc1ad39d2f" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:14.402335 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_0fd5cd6b3a5c4d0598149d887222ad33 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:14.403767 2609 leader_election.cc:341] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:14.404282 2763 leader_election.cc:341] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 5bf856a05dff4fb4943fe8e1d9b7f385
I20260927 23:28:14.417423 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_71487b69136749d4abdcb15306ae8daf"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:14.419052 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_71487b69136749d4abdcb15306ae8daf in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:14.437476 3070 tablet_bootstrap.cc:654] T 5c87d497da884544920b3be9c78daccf P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:14.450433 2966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56766ed12a7c4a668a7e08fc1ad39d2f" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:14.452330 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eff8f3e5ef74e93b077b5e5bfb139c6" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:14.453126 2971 raft_consensus.cc:2379] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
W20260927 23:28:14.454366 2761 leader_election.cc:341] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: 5bf856a05dff4fb4943fe8e1d9b7f385
I20260927 23:28:14.454797 2761 leader_election.cc:302] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:14.455916 2761 leader_election.cc:302] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:14.456928 3897 raft_consensus.cc:2726] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:14.457510 3897 raft_consensus.cc:3037] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:14.464639 3072 tablet_bootstrap.cc:492] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:14.465102 3072 ts_tablet_manager.cc:1397] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.218s user 0.009s sys 0.005s
I20260927 23:28:14.466917 3897 raft_consensus.cc:2726] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:14.468456 2608 leader_election.cc:341] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:14.468915 2608 leader_election.cc:302] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:14.469564 3796 raft_consensus.cc:2726] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:14.470805 3572 raft_consensus.cc:482] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:14.472580 3572 raft_consensus.cc:504] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.474678 3572 leader_election.cc:288] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:14.477473 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 155) took 5921 ms. Trace:
I20260927 23:28:14.477504 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbeeccc19a504e7984b293127766daf0" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:14.479056 2907 leader_election.cc:341] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:14.494526 3072 raft_consensus.cc:348] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.494442 3336 rpcz_store.cc:276] 0927 23:28:08.555586 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.555679 (+ 93us) service_pool.cc:227] Handling call
0927 23:28:14.234267 (+5678588us) ts_tablet_manager.cc:1145] Deleting tablet 3f71f948891b4d7fa95c3a1fbd92ce1e
0927 23:28:14.234324 (+ 57us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.296589 (+ 62265us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:14.375207 (+ 78618us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:14.477413 (+102206us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5678024}
I20260927 23:28:14.496320 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 3f71f948891b4d7fa95c3a1fbd92ce1e (table default.Table_047c5f89196f446e86279aae36cf7f99 [id=2aecf7c487414df1985aa5f284224515]) successfully deleted
I20260927 23:28:14.499006 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 156) took 5941 ms. Trace:
I20260927 23:28:14.499377 3336 rpcz_store.cc:276] 0927 23:28:08.557333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.557417 (+ 84us) service_pool.cc:227] Handling call
0927 23:28:14.497421 (+5940004us) ts_tablet_manager.cc:1145] Deleting tablet f58349edf0db40f98951e116fc2582b0
0927 23:28:14.497482 (+ 61us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.498940 (+ 1458us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5939410}
I20260927 23:28:14.499693 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbeeccc19a504e7984b293127766daf0" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:14.500979 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet f58349edf0db40f98951e116fc2582b0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f58349edf0db40f98951e116fc2582b0 already in progress: creating tablet
I20260927 23:28:14.502285 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 158) took 5754 ms. Trace:
I20260927 23:28:14.507565 3070 tablet_bootstrap.cc:492] T 5c87d497da884544920b3be9c78daccf P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:14.508046 3070 ts_tablet_manager.cc:1397] T 5c87d497da884544920b3be9c78daccf P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.246s user 0.016s sys 0.006s
I20260927 23:28:14.510488 3070 raft_consensus.cc:348] T 5c87d497da884544920b3be9c78daccf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.516469 3070 raft_consensus.cc:374] T 5c87d497da884544920b3be9c78daccf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:14.516777 3070 raft_consensus.cc:729] T 5c87d497da884544920b3be9c78daccf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:14.547443 2392 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_29404653bfbb4e5d8192bcb604084df0" } modify_external_catalogs: true
I20260927 23:28:14.518801 3070 consensus_queue.cc:260] T 5c87d497da884544920b3be9c78daccf P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.562393 3070 ts_tablet_manager.cc:1428] T 5c87d497da884544920b3be9c78daccf P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.054s user 0.006s sys 0.000s
I20260927 23:28:14.563321 3070 tablet_bootstrap.cc:492] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:14.563258 2392 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_29404653bfbb4e5d8192bcb604084df0" } modify_external_catalogs: true
I20260927 23:28:14.562832 2387 master_service.cc:946] ReplaceTablet: received request to replace tablet e81df7c7f1a04eb9904dd2a1a5a596c4 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:14.565654 3558 log.cc:1196] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/3c6e596517324300be30df8506a18904
I20260927 23:28:14.567046 3558 ts_tablet_manager.cc:1939] T 3c6e596517324300be30df8506a18904 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:14.572881 3070 tablet_bootstrap.cc:654] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:14.505123 2968 raft_consensus.cc:2454] T a708a517b744496c83a3aa759b3f1081 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
W20260927 23:28:14.505435 2906 leader_election.cc:341] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:14.573460 2906 leader_election.cc:302] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:14.574483 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 69084.1us
I20260927 23:28:14.574380 3912 raft_consensus.cc:2726] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:14.575577 2608 leader_election.cc:302] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:14.576689 3748 raft_consensus.cc:2781] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:14.577121 3748 raft_consensus.cc:686] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:14.577870 3748 consensus_queue.cc:237] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.585886 3336 rpcz_store.cc:276] 0927 23:28:08.747683 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.770216 (+ 22533us) service_pool.cc:227] Handling call
0927 23:28:14.501879 (+5731663us) ts_tablet_manager.cc:1145] Deleting tablet 8cd0972f635d4979abf5fd30e9b02b1d
0927 23:28:14.501933 (+ 54us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.502225 (+ 292us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5730961}
W20260927 23:28:14.587538 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 8cd0972f635d4979abf5fd30e9b02b1d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8cd0972f635d4979abf5fd30e9b02b1d already in progress: creating tablet
I20260927 23:28:14.588845 3336 tablet_replica.cc:330] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:14.589903 3336 raft_consensus.cc:2229] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:14.590617 3336 pending_rounds.cc:62] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d: Trying to abort 1 pending ops.
I20260927 23:28:14.591924 3336 pending_rounds.cc:68] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099716800065536 op_type: NO_OP noop_request { }
I20260927 23:28:14.599341 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_a66ee72e2abf4d8ca5f23c5ff0306640"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:14.611315 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 183) took 4064 ms. Trace:
I20260927 23:28:14.611930 3558 rpcz_store.cc:276] 0927 23:28:10.546826 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.546988 (+ 162us) service_pool.cc:227] Handling call
0927 23:28:14.075025 (+3528037us) ts_tablet_manager.cc:1145] Deleting tablet 3c6e596517324300be30df8506a18904
0927 23:28:14.075072 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.214876 (+139804us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:14.386417 (+171541us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:14.611233 (+224816us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3527463}
I20260927 23:28:14.619688 3167 tablet_bootstrap.cc:654] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:14.571374 3621 raft_consensus.cc:482] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:14.495250 3072 raft_consensus.cc:374] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:14.626302 3072 raft_consensus.cc:729] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:14.626385 3621 raft_consensus.cc:504] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.627138 3072 consensus_queue.cc:260] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.629642 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c87d497da884544920b3be9c78daccf" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:14.631258 2906 leader_election.cc:341] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:14.633352 3621 leader_election.cc:288] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:14.633767 3072 ts_tablet_manager.cc:1428] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.168s user 0.006s sys 0.001s
I20260927 23:28:14.634907 3072 tablet_bootstrap.cc:492] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:14.635185 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c87d497da884544920b3be9c78daccf" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:14.635855 2672 raft_consensus.cc:2454] T 5c87d497da884544920b3be9c78daccf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:14.635947 3336 raft_consensus.cc:2866] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:28:14.636667 3336 raft_consensus.cc:2258] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:14.637197 2907 leader_election.cc:302] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:14.638530 3621 raft_consensus.cc:2781] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:14.639068 3621 raft_consensus.cc:482] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:14.645987 3621 raft_consensus.cc:3037] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:14.647950 2377 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af], current schema version=2) failed: Not found: failed to reset TS proxy: No leader found: Tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 consensus state has no leader
I20260927 23:28:14.649369 3070 tablet_bootstrap.cc:492] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:14.650372 3070 ts_tablet_manager.cc:1397] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.087s user 0.013s sys 0.002s
I20260927 23:28:14.651849 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 181) took 4183 ms. Trace:
I20260927 23:28:14.656657 3558 rpcz_store.cc:276] 0927 23:28:10.431845 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.456275 (+ 24430us) service_pool.cc:227] Handling call
0927 23:28:14.614333 (+4158058us) ts_tablet_manager.cc:1145] Deleting tablet 26007ba8846f4b08b337c3ce03782f4a
0927 23:28:14.614457 (+ 124us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.615618 (+ 1161us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4044471}
I20260927 23:28:14.662480 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 186) took 3669 ms. Trace:
I20260927 23:28:14.658898 3621 raft_consensus.cc:504] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.613719 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 3c6e596517324300be30df8506a18904 (table default.Table_585d1f792a1b42e7ae86d5ecb5edaff2 [id=ff09406f8316449ead4cffa47a2095b8]) successfully deleted
W20260927 23:28:14.669426 2379 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 55773.8us
W20260927 23:28:14.671124 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 26007ba8846f4b08b337c3ce03782f4a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26007ba8846f4b08b337c3ce03782f4a already in progress: creating tablet
I20260927 23:28:14.671397 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c87d497da884544920b3be9c78daccf" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:14.671849 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a66ee72e2abf4d8ca5f23c5ff0306640 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:14.672946 2906 leader_election.cc:341] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:14.686407 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c87d497da884544920b3be9c78daccf" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:14.687081 2672 raft_consensus.cc:3037] T 5c87d497da884544920b3be9c78daccf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:14.694830 2672 raft_consensus.cc:2454] T 5c87d497da884544920b3be9c78daccf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:14.699446 3072 tablet_bootstrap.cc:654] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:14.699048 3796 consensus_queue.cc:1035] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:14.707471 2907 leader_election.cc:302] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:14.708725 3912 raft_consensus.cc:2781] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:14.709177 3912 raft_consensus.cc:686] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:14.709965 3912 consensus_queue.cc:237] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.714239 3070 raft_consensus.cc:348] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.714942 3070 raft_consensus.cc:374] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:14.715181 3070 raft_consensus.cc:729] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:14.715826 3621 leader_election.cc:288] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:14.663509 3558 rpcz_store.cc:276] 0927 23:28:10.992967 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:11.021778 (+ 28811us) service_pool.cc:227] Handling call
0927 23:28:14.661586 (+3639808us) ts_tablet_manager.cc:1145] Deleting tablet c848ed8a221a4b799529ccc0d1830893
0927 23:28:14.661720 (+ 134us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.662347 (+ 627us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3628334}
I20260927 23:28:14.717566 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 194) took 2619 ms. Trace:
I20260927 23:28:14.656046 3336 ts_tablet_manager.cc:1905] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:14.717937 3558 rpcz_store.cc:276] 0927 23:28:12.098189 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.103326 (+ 5137us) service_pool.cc:227] Handling call
0927 23:28:14.717160 (+2613834us) ts_tablet_manager.cc:1145] Deleting tablet 761a67c27c7a49a3903ed6f2bf5b96d3
0927 23:28:14.717207 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.717503 (+ 296us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2613031}
W20260927 23:28:14.720228 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet c848ed8a221a4b799529ccc0d1830893 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c848ed8a221a4b799529ccc0d1830893 already in progress: creating tablet
W20260927 23:28:14.721940 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 761a67c27c7a49a3903ed6f2bf5b96d3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 761a67c27c7a49a3903ed6f2bf5b96d3 already in progress: creating tablet
I20260927 23:28:14.743219 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 197) took 2558 ms. Trace:
I20260927 23:28:14.743805 3558 rpcz_store.cc:276] 0927 23:28:12.185258 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.185816 (+ 558us) service_pool.cc:227] Handling call
0927 23:28:14.719058 (+2533242us) ts_tablet_manager.cc:1145] Deleting tablet 7a5274b56b814c029e02c16243a3e663
0927 23:28:14.719101 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.743147 (+ 24046us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2532453}
I20260927 23:28:14.745541 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 198) took 2560 ms. Trace:
I20260927 23:28:14.746768 3621 raft_consensus.cc:482] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:14.747303 3621 raft_consensus.cc:504] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.747891 3167 tablet_bootstrap.cc:492] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:14.749558 3167 ts_tablet_manager.cc:1397] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.449s user 0.010s sys 0.002s
W20260927 23:28:14.755506 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 7a5274b56b814c029e02c16243a3e663 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a5274b56b814c029e02c16243a3e663 already in progress: creating tablet
I20260927 23:28:14.715785 3070 consensus_queue.cc:260] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.771045 3072 tablet_bootstrap.cc:492] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:14.773793 3072 ts_tablet_manager.cc:1397] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.139s user 0.008s sys 0.004s
I20260927 23:28:14.774636 3167 raft_consensus.cc:348] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.775461 3167 raft_consensus.cc:374] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:14.775761 3167 raft_consensus.cc:729] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:14.776531 3072 raft_consensus.cc:348] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.776576 3167 consensus_queue.cc:260] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.777235 3072 raft_consensus.cc:374] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:14.777474 3072 raft_consensus.cc:729] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:14.778048 3072 consensus_queue.cc:260] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.780802 3558 rpcz_store.cc:276] 0927 23:28:12.185585 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.186777 (+ 1192us) service_pool.cc:227] Handling call
0927 23:28:14.745130 (+2558353us) ts_tablet_manager.cc:1145] Deleting tablet 972763afd57847269fafadbdd2a5f3d6
0927 23:28:14.745179 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.745484 (+ 305us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2557688}
I20260927 23:28:14.781358 3072 ts_tablet_manager.cc:1428] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260927 23:28:14.782595 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 972763afd57847269fafadbdd2a5f3d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 972763afd57847269fafadbdd2a5f3d6 already in progress: creating tablet
I20260927 23:28:14.783885 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 196) took 2599 ms. Trace:
I20260927 23:28:14.784235 3558 rpcz_store.cc:276] 0927 23:28:12.184893 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.187595 (+ 2702us) service_pool.cc:227] Handling call
0927 23:28:14.783474 (+2595879us) ts_tablet_manager.cc:1145] Deleting tablet bc9fb7e66b6b45c29688d9379e24d95d
0927 23:28:14.783521 (+ 47us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.783828 (+ 307us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2595324}
I20260927 23:28:14.810741 3621 leader_election.cc:288] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:14.814745 3072 tablet_bootstrap.cc:492] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:14.798293 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56766ed12a7c4a668a7e08fc1ad39d2f" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:14.804807 3897 raft_consensus.cc:482] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:14.825282 3897 raft_consensus.cc:504] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.828006 3897 leader_election.cc:288] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:14.829005 2906 leader_election.cc:341] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:14.828271 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c87484b8b9742faa683f13d37308e58" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:14.829964 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c87484b8b9742faa683f13d37308e58" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:28:14.830687 2763 leader_election.cc:341] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:14.830963 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet bc9fb7e66b6b45c29688d9379e24d95d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc9fb7e66b6b45c29688d9379e24d95d already in progress: creating tablet
W20260927 23:28:14.831292 2761 leader_election.cc:341] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:14.831652 2761 leader_election.cc:302] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:14.832325 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 205) took 2179 ms. Trace:
I20260927 23:28:14.832867 3897 raft_consensus.cc:2726] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:14.834481 3072 tablet_bootstrap.cc:654] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:14.833788 3167 ts_tablet_manager.cc:1428] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.081s user 0.005s sys 0.000s
W20260927 23:28:14.853693 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 97661.6us
I20260927 23:28:14.858923 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56766ed12a7c4a668a7e08fc1ad39d2f" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:14.860073 2672 raft_consensus.cc:2454] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:14.865038 2907 leader_election.cc:302] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:14.874496 3796 consensus_queue.cc:1035] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:14.876272 3070 ts_tablet_manager.cc:1428] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.225s user 0.006s sys 0.001s
I20260927 23:28:14.877802 3070 tablet_bootstrap.cc:492] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:14.832659 3558 rpcz_store.cc:276] 0927 23:28:12.653274 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.653435 (+ 161us) service_pool.cc:227] Handling call
0927 23:28:14.831936 (+2178501us) ts_tablet_manager.cc:1145] Deleting tablet fc60d0c7a3794607814393f033238052
0927 23:28:14.831984 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.832246 (+ 262us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2177954}
I20260927 23:28:14.892469 3167 tablet_bootstrap.cc:492] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
W20260927 23:28:14.897073 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet fc60d0c7a3794607814393f033238052 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc60d0c7a3794607814393f033238052 already in progress: creating tablet
I20260927 23:28:14.898414 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 209) took 1961 ms. Trace:
I20260927 23:28:14.907539 3558 rpcz_store.cc:276] 0927 23:28:12.936839 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.936960 (+ 121us) service_pool.cc:227] Handling call
0927 23:28:14.898025 (+1961065us) ts_tablet_manager.cc:1145] Deleting tablet 10fe6b0a4c614b77ae95b200b6a12843
0927 23:28:14.898073 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.898336 (+ 263us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1960448}
W20260927 23:28:14.916333 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 10fe6b0a4c614b77ae95b200b6a12843 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 10fe6b0a4c614b77ae95b200b6a12843 already in progress: creating tablet
I20260927 23:28:14.928180 3336 ts_tablet_manager.cc:1918] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:14.931502 3167 tablet_bootstrap.cc:654] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:14.932602 3336 log.cc:1196] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/cfbbe17f54d1418b95efc3943db0b6c1
I20260927 23:28:14.933610 3070 tablet_bootstrap.cc:654] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:14.935302 3336 ts_tablet_manager.cc:1939] T cfbbe17f54d1418b95efc3943db0b6c1 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:14.935232 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 212) took 1808 ms. Trace:
I20260927 23:28:14.935595 3558 rpcz_store.cc:276] 0927 23:28:13.126816 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.126930 (+ 114us) service_pool.cc:227] Handling call
0927 23:28:14.920245 (+1793315us) ts_tablet_manager.cc:1145] Deleting tablet 9af9b87abf1f4f78b3140a91f2525066
0927 23:28:14.934864 (+ 14619us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.935166 (+ 302us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1792734}
I20260927 23:28:14.939716 3558 tablet_replica.cc:330] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:14.941112 3558 raft_consensus.cc:2229] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus shutting down.
W20260927 23:28:14.936952 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 9af9b87abf1f4f78b3140a91f2525066 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9af9b87abf1f4f78b3140a91f2525066 already in progress: creating tablet
I20260927 23:28:14.943648 3558 raft_consensus.cc:2258] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:14.939278 3866 raft_consensus.cc:2781] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:14.949352 3866 raft_consensus.cc:482] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:14.949663 3866 raft_consensus.cc:3037] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:14.967535 3558 ts_tablet_manager.cc:1905] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:14.971176 3072 tablet_bootstrap.cc:492] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:14.971657 3072 ts_tablet_manager.cc:1397] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.157s user 0.010s sys 0.005s
I20260927 23:28:14.974006 3072 raft_consensus.cc:348] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.981220 3748 raft_consensus.cc:482] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:14.981768 3748 raft_consensus.cc:504] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.983752 3748 leader_election.cc:288] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:14.985503 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1890852164d492db3540a97847888fd" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:14.987463 2609 leader_election.cc:341] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:14.988437 3167 tablet_bootstrap.cc:492] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:14.989878 3167 ts_tablet_manager.cc:1397] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.098s user 0.004s sys 0.007s
I20260927 23:28:14.993388 3072 raft_consensus.cc:374] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:14.994107 3072 raft_consensus.cc:729] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:14.995358 3072 consensus_queue.cc:260] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:14.988973 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1890852164d492db3540a97847888fd" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:28:15.009793 2608 leader_election.cc:341] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:15.010239 2608 leader_election.cc:302] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:14.983232 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 159) took 6171 ms. Trace:
I20260927 23:28:15.014132 3928 raft_consensus.cc:2726] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:15.014263 3336 rpcz_store.cc:276] 0927 23:28:08.811795 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.815486 (+ 3691us) service_pool.cc:227] Handling call
0927 23:28:14.588632 (+5773146us) ts_tablet_manager.cc:1145] Deleting tablet cfbbe17f54d1418b95efc3943db0b6c1
0927 23:28:14.588709 (+ 77us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:14.731668 (+142959us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:14.927900 (+196232us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:14.983161 (+ 55261us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5772575}
I20260927 23:28:15.015669 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 163) took 5380 ms. Trace:
I20260927 23:28:15.015803 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet cfbbe17f54d1418b95efc3943db0b6c1 (table default.Table_96276d84ee11493d89908e4ff16dc7fe [id=f21b53d8e90a48bc9cb7f6268c946cc1]) successfully deleted
I20260927 23:28:15.015970 3336 rpcz_store.cc:276] 0927 23:28:09.635572 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.636350 (+ 778us) service_pool.cc:227] Handling call
0927 23:28:15.015256 (+5378906us) ts_tablet_manager.cc:1145] Deleting tablet a1890852164d492db3540a97847888fd
0927 23:28:15.015327 (+ 71us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:15.015618 (+ 291us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5378306}
I20260927 23:28:15.017258 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 162) took 5387 ms. Trace:
W20260927 23:28:15.017270 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet a1890852164d492db3540a97847888fd because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1890852164d492db3540a97847888fd already in progress: creating tablet
I20260927 23:28:15.017167 3897 consensus_queue.cc:1035] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:15.017668 3336 rpcz_store.cc:276] 0927 23:28:09.630075 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.634885 (+ 4810us) service_pool.cc:227] Handling call
0927 23:28:15.016801 (+5381916us) ts_tablet_manager.cc:1145] Deleting tablet 8b891bab5fb343b0bb94641aea7662f0
0927 23:28:15.016856 (+ 55us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:15.017199 (+ 343us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5374969}
I20260927 23:28:15.018814 3336 tablet_replica.cc:330] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
W20260927 23:28:15.019030 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 8b891bab5fb343b0bb94641aea7662f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 8b891bab5fb343b0bb94641aea7662f0 already in progress: creating tablet
I20260927 23:28:15.019636 3336 raft_consensus.cc:2229] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:14.972186 3866 raft_consensus.cc:504] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.022993 3072 ts_tablet_manager.cc:1428] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.051s user 0.003s sys 0.003s
I20260927 23:28:15.026641 3072 tablet_bootstrap.cc:492] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:15.039453 3072 tablet_bootstrap.cc:654] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:15.042711 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56766ed12a7c4a668a7e08fc1ad39d2f" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:15.047896 3070 tablet_bootstrap.cc:492] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:15.059134 3070 ts_tablet_manager.cc:1397] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.181s user 0.005s sys 0.006s
I20260927 23:28:15.067200 3070 raft_consensus.cc:348] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.068230 2873 heartbeater.cc:347] Connected to a master server at 127.2.57.60:41571
I20260927 23:28:15.070732 3070 raft_consensus.cc:374] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:15.071220 3070 raft_consensus.cc:729] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
W20260927 23:28:15.044346 2906 leader_election.cc:341] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:15.076970 2761 consensus_peers.cc:487] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 -> Peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Couldn't send request to peer de216a4a8e4b4e1b857d8647a3b05f6d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2411d7c53a154fafad9edda9fa75bb36. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:15.077514 3866 leader_election.cc:288] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:15.069484 3941 tablet.cc:1722] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260927 23:28:15.020129 3336 raft_consensus.cc:2258] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:15.085686 3336 ts_tablet_manager.cc:1905] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:15.071221 3942 tablet.cc:1722] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260927 23:28:15.104465 3167 raft_consensus.cc:348] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.105443 3167 raft_consensus.cc:374] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:15.105765 3167 raft_consensus.cc:729] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:15.106524 3167 consensus_queue.cc:260] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.077863 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56766ed12a7c4a668a7e08fc1ad39d2f" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:15.094136 3945 tablet.cc:1722] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260927 23:28:15.124609 2674 raft_consensus.cc:3037] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:15.124850 3070 consensus_queue.cc:260] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.127311 3070 ts_tablet_manager.cc:1428] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.062s user 0.003s sys 0.002s
I20260927 23:28:15.128165 3070 tablet_bootstrap.cc:492] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:15.133481 2674 raft_consensus.cc:2454] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:15.135003 2907 leader_election.cc:302] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:15.136122 3866 raft_consensus.cc:2781] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:15.136571 3866 raft_consensus.cc:686] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:15.137316 3866 consensus_queue.cc:237] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.143045 3072 tablet_bootstrap.cc:492] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:15.143493 3072 ts_tablet_manager.cc:1397] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.117s user 0.014s sys 0.012s
I20260927 23:28:15.147081 3072 raft_consensus.cc:348] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.147775 3072 raft_consensus.cc:374] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:15.148025 3072 raft_consensus.cc:729] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:15.152908 3072 consensus_queue.cc:260] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.153889 3572 raft_consensus.cc:482] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:15.154296 3572 raft_consensus.cc:504] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.155115 3748 raft_consensus.cc:482] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:15.155515 3748 raft_consensus.cc:504] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.156565 3572 leader_election.cc:288] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:15.157349 3748 leader_election.cc:288] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:15.158660 3072 ts_tablet_manager.cc:1428] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260927 23:28:15.159113 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49883297d2b046b99f2ec1343c276ff9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:28:15.162258 2608 leader_election.cc:341] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:15.168275 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4a9d69d75c4bfbb43e558b83b460d1" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:15.169867 2907 leader_election.cc:341] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:15.170989 3072 tablet_bootstrap.cc:492] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:15.185824 3070 tablet_bootstrap.cc:654] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:15.205034 3336 ts_tablet_manager.cc:1918] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:15.206423 3336 log.cc:1196] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/fd0115095a82489a8aeb941efcc8a979
I20260927 23:28:15.158119 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4a9d69d75c4bfbb43e558b83b460d1" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:15.207619 3866 raft_consensus.cc:482] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:15.161293 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49883297d2b046b99f2ec1343c276ff9" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:15.208164 3866 raft_consensus.cc:504] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:15.209345 2906 leader_election.cc:341] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:15.209537 2609 leader_election.cc:341] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:15.209728 2906 leader_election.cc:302] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:15.209959 2609 leader_election.cc:302] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:15.210952 3748 raft_consensus.cc:2726] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:15.211310 3866 leader_election.cc:288] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:15.211709 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "371842420a6048fca3cbc02d18f74ffc" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:15.212383 3866 raft_consensus.cc:2726] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:15.207286 3336 ts_tablet_manager.cc:1939] T fd0115095a82489a8aeb941efcc8a979 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:15.212098 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "371842420a6048fca3cbc02d18f74ffc" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:15.213182 2906 leader_election.cc:341] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:15.223440 3167 ts_tablet_manager.cc:1428] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.233s user 0.006s sys 0.000s
I20260927 23:28:15.219864 2674 raft_consensus.cc:2454] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:15.223264 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 161) took 5596 ms. Trace:
I20260927 23:28:15.224818 3336 rpcz_store.cc:276] 0927 23:28:09.627175 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:09.629445 (+ 2270us) service_pool.cc:227] Handling call
0927 23:28:15.018645 (+5389200us) ts_tablet_manager.cc:1145] Deleting tablet fd0115095a82489a8aeb941efcc8a979
0927 23:28:15.018705 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:15.164123 (+145418us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:15.204781 (+ 40658us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:15.223194 (+ 18413us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5376249}
I20260927 23:28:15.225328 2907 leader_election.cc:302] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:15.226078 3572 raft_consensus.cc:2781] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:15.226073 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet fd0115095a82489a8aeb941efcc8a979 (table default.Table_0df33121d2eb4445ac8b8c6a9ee44d24 [id=0a5810627a964e7caf66c69e7c3ec9b4]) successfully deleted
I20260927 23:28:15.226388 3572 raft_consensus.cc:482] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:15.226672 3572 raft_consensus.cc:3037] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:15.226913 3072 tablet_bootstrap.cc:654] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:15.235912 3572 raft_consensus.cc:504] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.237056 3070 tablet_bootstrap.cc:492] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:15.237496 3070 ts_tablet_manager.cc:1397] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.109s user 0.011s sys 0.000s
I20260927 23:28:15.240667 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 182) took 4831 ms. Trace:
I20260927 23:28:15.241005 3336 rpcz_store.cc:276] 0927 23:28:10.409623 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.438341 (+ 28718us) service_pool.cc:227] Handling call
0927 23:28:15.233288 (+4794947us) ts_tablet_manager.cc:1145] Deleting tablet ee5d073c8df84f0884dc0f0d88eeed9f
0927 23:28:15.233353 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:15.240593 (+ 7240us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4781873}
W20260927 23:28:15.242429 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet ee5d073c8df84f0884dc0f0d88eeed9f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ee5d073c8df84f0884dc0f0d88eeed9f already in progress: creating tablet
I20260927 23:28:15.244797 3070 raft_consensus.cc:348] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.245481 3070 raft_consensus.cc:374] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:15.245720 3070 raft_consensus.cc:729] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:15.246753 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 185) took 4538 ms. Trace:
I20260927 23:28:15.247179 3336 rpcz_store.cc:276] 0927 23:28:10.707968 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.708086 (+ 118us) service_pool.cc:227] Handling call
0927 23:28:15.246291 (+4538205us) ts_tablet_manager.cc:1145] Deleting tablet 26007ba8846f4b08b337c3ce03782f4a
0927 23:28:15.246350 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:15.246686 (+ 336us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4537668}
I20260927 23:28:15.239784 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "371842420a6048fca3cbc02d18f74ffc" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:15.248194 3336 tablet_replica.cc:330] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:15.249008 3336 raft_consensus.cc:2229] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus shutting down.
W20260927 23:28:15.249085 2906 leader_election.cc:341] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:15.249119 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 26007ba8846f4b08b337c3ce03782f4a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26007ba8846f4b08b337c3ce03782f4a already in progress: creating tablet
I20260927 23:28:15.249470 3336 raft_consensus.cc:2258] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:15.249581 3572 leader_election.cc:288] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:15.251350 3336 ts_tablet_manager.cc:1905] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:15.254938 3070 consensus_queue.cc:260] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.262075 3070 ts_tablet_manager.cc:1428] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.024s user 0.008s sys 0.000s
I20260927 23:28:15.263535 3167 tablet_bootstrap.cc:492] T 371842420a6048fca3cbc02d18f74ffc P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:15.263804 2721 heartbeater.cc:347] Connected to a master server at 127.2.57.60:41571
I20260927 23:28:15.282265 3072 tablet_bootstrap.cc:492] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:15.282763 3072 ts_tablet_manager.cc:1397] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.112s user 0.009s sys 0.005s
I20260927 23:28:15.294668 3072 raft_consensus.cc:348] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.295429 3072 raft_consensus.cc:374] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:15.295723 3072 raft_consensus.cc:729] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:15.296434 3072 consensus_queue.cc:260] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.304598 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "371842420a6048fca3cbc02d18f74ffc" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:15.305243 2674 raft_consensus.cc:3037] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:15.308935 3072 ts_tablet_manager.cc:1428] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.026s user 0.004s sys 0.004s
I20260927 23:28:15.309863 3072 tablet_bootstrap.cc:492] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:15.315316 2674 raft_consensus.cc:2454] T 371842420a6048fca3cbc02d18f74ffc P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:15.324329 3866 consensus_queue.cc:1035] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:15.325623 3167 tablet_bootstrap.cc:654] T 371842420a6048fca3cbc02d18f74ffc P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:15.326063 2907 leader_election.cc:302] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:15.329701 3866 raft_consensus.cc:2781] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:15.330188 3866 raft_consensus.cc:686] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:15.331045 3866 consensus_queue.cc:237] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.334740 3070 tablet_bootstrap.cc:492] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:15.349802 2947 tablet_service.cc:1467] Processing CreateTablet for tablet fafd1c9280a545ada3cdb040c22b3786 (DEFAULT_TABLE table=default.Table_cd24927b2fec4eee916e402106212f33 [id=8cf1f3bb6f7d41beb15571d3b8da05b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.350540 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 4f7667f8cf3b4996b07eb815aabb4f47 (DEFAULT_TABLE table=default.Table_b41d59628db149279b4f71e292a05f98 [id=79e5739a9141499f8beb87a171525430]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.351778 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fafd1c9280a545ada3cdb040c22b3786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.359990 2807 tablet_service.cc:1467] Processing CreateTablet for tablet 4f7667f8cf3b4996b07eb815aabb4f47 (DEFAULT_TABLE table=default.Table_b41d59628db149279b4f71e292a05f98 [id=79e5739a9141499f8beb87a171525430]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.361150 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f7667f8cf3b4996b07eb815aabb4f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.363152 3336 ts_tablet_manager.cc:1918] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:15.363541 3336 log.cc:1196] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/bbeeccc19a504e7984b293127766daf0
I20260927 23:28:15.363687 3070 tablet_bootstrap.cc:654] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:15.363438 2800 tablet_service.cc:1467] Processing CreateTablet for tablet d2cb1b536b4540a2849643309dd27185 (DEFAULT_TABLE table=default.Table_1ad5ebc0110d45a280c889a7b7670506 [id=3ef6d3cab6834e859b1239737e03fe03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.364589 2800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2cb1b536b4540a2849643309dd27185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.365706 3072 tablet_bootstrap.cc:654] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:15.365151 2798 tablet_service.cc:1467] Processing CreateTablet for tablet fafd1c9280a545ada3cdb040c22b3786 (DEFAULT_TABLE table=default.Table_cd24927b2fec4eee916e402106212f33 [id=8cf1f3bb6f7d41beb15571d3b8da05b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.366658 2798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fafd1c9280a545ada3cdb040c22b3786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.376557 3558 ts_tablet_manager.cc:1918] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:15.377703 2649 tablet_service.cc:1467] Processing CreateTablet for tablet 4f7667f8cf3b4996b07eb815aabb4f47 (DEFAULT_TABLE table=default.Table_b41d59628db149279b4f71e292a05f98 [id=79e5739a9141499f8beb87a171525430]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.378759 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f7667f8cf3b4996b07eb815aabb4f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.381073 2651 tablet_service.cc:1467] Processing CreateTablet for tablet fafd1c9280a545ada3cdb040c22b3786 (DEFAULT_TABLE table=default.Table_cd24927b2fec4eee916e402106212f33 [id=8cf1f3bb6f7d41beb15571d3b8da05b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.382073 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fafd1c9280a545ada3cdb040c22b3786. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.387400 2937 tablet_service.cc:1467] Processing CreateTablet for tablet d2cb1b536b4540a2849643309dd27185 (DEFAULT_TABLE table=default.Table_1ad5ebc0110d45a280c889a7b7670506 [id=3ef6d3cab6834e859b1239737e03fe03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.388641 2937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2cb1b536b4540a2849643309dd27185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.389737 3928 consensus_queue.cc:1035] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:15.390182 2653 tablet_service.cc:1467] Processing CreateTablet for tablet d2cb1b536b4540a2849643309dd27185 (DEFAULT_TABLE table=default.Table_1ad5ebc0110d45a280c889a7b7670506 [id=3ef6d3cab6834e859b1239737e03fe03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.391224 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d2cb1b536b4540a2849643309dd27185. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.393800 3070 tablet_bootstrap.cc:492] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:15.394222 3070 ts_tablet_manager.cc:1397] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.060s user 0.013s sys 0.002s
I20260927 23:28:15.396957 3070 raft_consensus.cc:348] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.397644 3070 raft_consensus.cc:374] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:15.397903 3070 raft_consensus.cc:729] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:15.398722 3070 consensus_queue.cc:260] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.399222 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f7667f8cf3b4996b07eb815aabb4f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.401932 3070 ts_tablet_manager.cc:1428] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260927 23:28:15.402844 3070 tablet_bootstrap.cc:492] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:15.407815 2804 tablet_service.cc:1467] Processing CreateTablet for tablet b3cb46ee19924aa0acf7531e75a80ad4 (DEFAULT_TABLE table=default.Table_4f8442f0e39b47a292bf291549e64212 [id=578ab61235004c2b841d0c459b46bd85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.409399 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3cb46ee19924aa0acf7531e75a80ad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.410629 2943 tablet_service.cc:1467] Processing CreateTablet for tablet b3cb46ee19924aa0acf7531e75a80ad4 (DEFAULT_TABLE table=default.Table_4f8442f0e39b47a292bf291549e64212 [id=578ab61235004c2b841d0c459b46bd85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.411727 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3cb46ee19924aa0acf7531e75a80ad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.412002 2646 tablet_service.cc:1467] Processing CreateTablet for tablet b3cb46ee19924aa0acf7531e75a80ad4 (DEFAULT_TABLE table=default.Table_4f8442f0e39b47a292bf291549e64212 [id=578ab61235004c2b841d0c459b46bd85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.412082 3912 consensus_queue.cc:1035] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:15.413040 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3cb46ee19924aa0acf7531e75a80ad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.417955 3070 tablet_bootstrap.cc:654] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:15.448201 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:15.449283 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:15.450351 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:15.451014 2796 tablet_service.cc:1467] Processing CreateTablet for tablet d7dd7ce63f35444c83184b94e4898fdc (DEFAULT_TABLE table=default.Table_1458cf4426054ad583f77c955513541c [id=77ee9be006b84759b8d18e803b7d33a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.452226 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7dd7ce63f35444c83184b94e4898fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.460381 3558 log.cc:1196] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/3200d6bd5cb24ff5b1bbd59af9353dbe
I20260927 23:28:15.460517 3070 tablet_bootstrap.cc:492] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:15.461190 3558 ts_tablet_manager.cc:1939] T 3200d6bd5cb24ff5b1bbd59af9353dbe P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:15.461292 3070 ts_tablet_manager.cc:1397] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.059s user 0.009s sys 0.004s
I20260927 23:28:15.463403 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 214) took 1987 ms. Trace:
I20260927 23:28:15.465251 3070 raft_consensus.cc:348] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.465893 3070 raft_consensus.cc:374] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:15.466120 3070 raft_consensus.cc:729] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:15.468359 3070 consensus_queue.cc:260] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.470432 3070 ts_tablet_manager.cc:1428] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.009s user 0.004s sys 0.000s
I20260927 23:28:15.471364 3070 tablet_bootstrap.cc:492] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:15.474464 2644 tablet_service.cc:1467] Processing CreateTablet for tablet d7dd7ce63f35444c83184b94e4898fdc (DEFAULT_TABLE table=default.Table_1458cf4426054ad583f77c955513541c [id=77ee9be006b84759b8d18e803b7d33a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.475867 2644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7dd7ce63f35444c83184b94e4898fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.478739 3070 tablet_bootstrap.cc:654] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:15.482024 2392 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 78e099028f7847d1b3de7d437be97de0
I20260927 23:28:15.485085 2932 tablet_service.cc:1514] Processing DeleteTablet for tablet 78e099028f7847d1b3de7d437be97de0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:14 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:15.486641 2392 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 289) took 1096 ms. Trace:
I20260927 23:28:15.487303 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 78e099028f7847d1b3de7d437be97de0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:14 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:15.487032 2392 rpcz_store.cc:276] 0927 23:28:14.389853 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:14.547033 (+157180us) service_pool.cc:227] Handling call
0927 23:28:14.563870 (+ 16837us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:14.564649 (+ 779us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:28:14.564852 (+ 203us) catalog_manager.cc:2761] Locking tablets
0927 23:28:15.344953 (+780101us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:28:15.481523 (+136570us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:28:15.481773 (+ 250us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:15.481907 (+ 134us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:15.486578 (+ 4671us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:15.351885 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:15.352252 (+ 367us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:15.352277 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:28:15.352289 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:15.352632 (+ 343us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:15.352685 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:15.352708 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:15.352975 (+ 267us) tablet.cc:699] Row locks acquired
0927 23:28:15.352987 (+ 12us) write_op.cc:260] PREPARE: finished
0927 23:28:15.353111 (+ 124us) write_op.cc:270] Start()
0927 23:28:15.353204 (+ 93us) write_op.cc:276] Timestamp: P: 1790551695353043 usec, L: 0
0927 23:28:15.353220 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:15.353793 (+ 573us) log.cc:842] Serialized 753 byte log entry
0927 23:28:15.478769 (+124976us) op_driver.cc:444] REPLICATION: finished
0927 23:28:15.479480 (+ 711us) write_op.cc:301] APPLY: starting
0927 23:28:15.479539 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:15.479649 (+ 110us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:15.479825 (+ 176us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:15.480111 (+ 286us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:15.480244 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:15.480282 (+ 38us) write_op.cc:312] APPLY: finished
0927 23:28:15.480508 (+ 226us) log.cc:842] Serialized 29 byte log entry
0927 23:28:15.480664 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:15.480739 (+ 75us) write_op.cc:454] Released schema lock
0927 23:28:15.480770 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":546,"num_ops":2,"prepare.queue_time_us":4891,"prepare.run_cpu_time_us":2214,"prepare.run_wall_time_us":2215,"raft.queue_time_us":64127,"raft.run_cpu_time_us":917,"raft.run_wall_time_us":1915,"replication_time_us":125453}]]}
I20260927 23:28:15.491432 2801 tablet_service.cc:1514] Processing DeleteTablet for tablet 78e099028f7847d1b3de7d437be97de0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:14 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:15.510579 2387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet e81df7c7f1a04eb9904dd2a1a5a596c4
I20260927 23:28:15.513263 2932 tablet_service.cc:1514] Processing DeleteTablet for tablet e81df7c7f1a04eb9904dd2a1a5a596c4 with delete_type TABLET_DATA_DELETED (replaced by tablet e328482c065f442cbc285584c6ffade4) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:15.513895 2387 catalog_manager.cc:6565] ReplaceTablet: tablet e81df7c7f1a04eb9904dd2a1a5a596c4 deleted and replaced by tablet e328482c065f442cbc285584c6ffade4
W20260927 23:28:15.515854 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 78e099028f7847d1b3de7d437be97de0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78e099028f7847d1b3de7d437be97de0 already in progress: creating tablet
I20260927 23:28:15.516211 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet e81df7c7f1a04eb9904dd2a1a5a596c4 with delete_type TABLET_DATA_DELETED (replaced by tablet e328482c065f442cbc285584c6ffade4) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:15.519943 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af]). Expected version 2 got 0
I20260927 23:28:15.520684 2801 tablet_service.cc:1514] Processing DeleteTablet for tablet e81df7c7f1a04eb9904dd2a1a5a596c4 with delete_type TABLET_DATA_DELETED (replaced by tablet e328482c065f442cbc285584c6ffade4) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:15.521224 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 1 got 0
I20260927 23:28:15.528807 3167 tablet_bootstrap.cc:492] T 371842420a6048fca3cbc02d18f74ffc P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:15.529266 3167 ts_tablet_manager.cc:1397] T 371842420a6048fca3cbc02d18f74ffc P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.266s user 0.014s sys 0.000s
W20260927 23:28:15.533398 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet e81df7c7f1a04eb9904dd2a1a5a596c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e81df7c7f1a04eb9904dd2a1a5a596c4 already in progress: creating tablet
I20260927 23:28:15.533558 3558 rpcz_store.cc:276] 0927 23:28:13.476456 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.480765 (+ 4309us) service_pool.cc:227] Handling call
0927 23:28:14.939419 (+1458654us) ts_tablet_manager.cc:1145] Deleting tablet 3200d6bd5cb24ff5b1bbd59af9353dbe
0927 23:28:14.939547 (+ 128us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:15.249040 (+309493us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:15.376247 (+127207us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:15.463336 (+ 87089us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1457985}
I20260927 23:28:15.535172 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 3200d6bd5cb24ff5b1bbd59af9353dbe (table default.Table_e57880e3ffa64d3b876813b8c2246002 [id=805a37d784f64efe9c46c985e5c853f1]) successfully deleted
I20260927 23:28:15.364168 3336 ts_tablet_manager.cc:1939] T bbeeccc19a504e7984b293127766daf0 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:15.546952 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 186) took 4833 ms. Trace:
I20260927 23:28:15.547461 3336 rpcz_store.cc:276] 0927 23:28:10.713929 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.714491 (+ 562us) service_pool.cc:227] Handling call
0927 23:28:15.248018 (+4533527us) ts_tablet_manager.cc:1145] Deleting tablet bbeeccc19a504e7984b293127766daf0
0927 23:28:15.248077 (+ 59us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:15.328458 (+ 80381us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:15.362897 (+ 34439us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:15.544921 (+182024us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4533038}
W20260927 23:28:15.559366 2609 consensus_peers.cc:487] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:15.584378 3621 raft_consensus.cc:482] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:15.584848 3621 raft_consensus.cc:504] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.586692 3621 leader_election.cc:288] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:15.589323 2906 leader_election.cc:341] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Not found: Tablet not found: 3c6e596517324300be30df8506a18904
I20260927 23:28:15.591738 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 216b4311431b44bca04b6a681b35537b (DEFAULT_TABLE table=default.Table_a3208426010f46cf820de6a4dbba0099 [id=7efc17a625c6488e960a30fea44f1f0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.593794 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 216b4311431b44bca04b6a681b35537b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.596818 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 215) took 2069 ms. Trace:
I20260927 23:28:15.597316 3558 rpcz_store.cc:276] 0927 23:28:13.527150 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.527286 (+ 136us) service_pool.cc:227] Handling call
0927 23:28:15.596509 (+2069223us) ts_tablet_manager.cc:1145] Deleting tablet cfbbe17f54d1418b95efc3943db0b6c1
0927 23:28:15.596555 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:15.596758 (+ 203us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2068532}
I20260927 23:28:15.599326 3558 tablet_replica.cc:330] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:15.600096 3558 raft_consensus.cc:2229] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:15.600628 3558 pending_rounds.cc:62] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51: Trying to abort 1 pending ops.
I20260927 23:28:15.600919 3558 pending_rounds.cc:68] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099728269565952 op_type: NO_OP noop_request { }
W20260927 23:28:15.604650 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet cfbbe17f54d1418b95efc3943db0b6c1 because the tablet was not found. No further retry: Not found: Tablet not found: cfbbe17f54d1418b95efc3943db0b6c1
I20260927 23:28:15.605705 3072 tablet_bootstrap.cc:492] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:15.606184 3072 ts_tablet_manager.cc:1397] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.296s user 0.001s sys 0.010s
W20260927 23:28:15.613626 3913 lock_manager.cc:586] Waited 1 seconds to obtain row lock on key '<redacted>' (key hash 17596536596126568050) tablet 00000000000000000000000000000000 cur holder 0x7b5800071700
I20260927 23:28:15.614202 3336 tablet_replica.cc:330] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:15.615195 3336 raft_consensus.cc:2229] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:15.615777 3336 raft_consensus.cc:2258] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:15.618320 3336 ts_tablet_manager.cc:1905] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:15.625948 3965 raft_consensus.cc:482] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:15.626664 3965 raft_consensus.cc:504] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.632134 2393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 3a8dbc1165ac4f5b8ed792f6064cce6f
I20260927 23:28:15.633685 2393 catalog_manager.cc:6565] ReplaceTablet: tablet 3a8dbc1165ac4f5b8ed792f6064cce6f deleted and replaced by tablet b9c86c7d874a49178c7b2cd7316afa14
I20260927 23:28:15.634038 2393 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:51972 (request call id 284) took 1561 ms. Trace:
I20260927 23:28:15.636914 3558 raft_consensus.cc:2866] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:28:15.637410 3558 raft_consensus.cc:2258] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:15.634312 2393 rpcz_store.cc:276] 0927 23:28:14.072438 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:14.135570 (+ 63132us) service_pool.cc:227] Handling call
0927 23:28:14.304796 (+169226us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b5b81 00007f8bb3c10d37 00007f8bb3c10cc2 00007f8bb3c10bf1 00007f8bb3c10b6c 00007f8bb3c0fe22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:15.633966 (+1329170us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:15.425365 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:15.425850 (+ 485us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:15.425879 (+ 29us) write_op.cc:435] Acquired schema lock
0927 23:28:15.425894 (+ 15us) tablet.cc:660] Decoding operations
0927 23:28:15.426263 (+ 369us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:15.426307 (+ 44us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:15.426332 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:15.426559 (+ 227us) tablet.cc:699] Row locks acquired
0927 23:28:15.426573 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:15.426707 (+ 134us) write_op.cc:270] Start()
0927 23:28:15.427201 (+ 494us) write_op.cc:276] Timestamp: P: 1790551695426630 usec, L: 0
0927 23:28:15.427223 (+ 22us) op_driver.cc:331] REPLICATION: starting
0927 23:28:15.427868 (+ 645us) log.cc:842] Serialized 619 byte log entry
0927 23:28:15.615342 (+187474us) op_driver.cc:444] REPLICATION: finished
0927 23:28:15.621586 (+ 6244us) write_op.cc:301] APPLY: starting
0927 23:28:15.621645 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:15.621762 (+ 117us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:15.621887 (+ 125us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:15.622343 (+ 456us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:15.622566 (+ 223us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:15.622613 (+ 47us) write_op.cc:312] APPLY: finished
0927 23:28:15.622833 (+ 220us) log.cc:842] Serialized 29 byte log entry
0927 23:28:15.629208 (+ 6375us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:15.629291 (+ 83us) write_op.cc:454] Released schema lock
0927 23:28:15.629326 (+ 35us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":384774400,"child_traces":[["op",{"apply.queue_time_us":6043,"num_ops":2,"prepare.queue_time_us":15269,"prepare.run_cpu_time_us":2438,"prepare.run_wall_time_us":36960,"replication_time_us":188026}]]}
I20260927 23:28:15.639096 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 1 got 0
I20260927 23:28:15.640582 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:15.641211 2393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_cccab8d895c740e0b88b2709527516e3" } new_table_name: "default.Table_f45431c208e642ef9f406af42ff7d455" modify_external_catalogs: true
I20260927 23:28:15.630612 3167 raft_consensus.cc:348] T 371842420a6048fca3cbc02d18f74ffc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.644271 3167 raft_consensus.cc:374] T 371842420a6048fca3cbc02d18f74ffc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:15.641347 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af]). Expected version 2 got 0
I20260927 23:28:15.644563 3167 raft_consensus.cc:729] T 371842420a6048fca3cbc02d18f74ffc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:15.645675 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:15.645622 3167 consensus_queue.cc:260] T 371842420a6048fca3cbc02d18f74ffc P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.646612 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:15.647173 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
I20260927 23:28:15.647753 3167 ts_tablet_manager.cc:1428] T 371842420a6048fca3cbc02d18f74ffc P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.118s user 0.005s sys 0.000s
I20260927 23:28:15.659883 2947 tablet_service.cc:1467] Processing CreateTablet for tablet d7dd7ce63f35444c83184b94e4898fdc (DEFAULT_TABLE table=default.Table_1458cf4426054ad583f77c955513541c [id=77ee9be006b84759b8d18e803b7d33a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.668201 3558 ts_tablet_manager.cc:1905] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:15.672392 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 4136d27807db40649d915422d7c10f6d (DEFAULT_TABLE table=default.Table_11f58699a34140ff8ad27a712df72ab2 [id=bd2a9767c4904134a96dc5c2a75159aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.673487 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4136d27807db40649d915422d7c10f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.678975 2794 tablet_service.cc:1467] Processing CreateTablet for tablet 46e23759c43c49ba85a3ba8c47131850 (DEFAULT_TABLE table=default.Table_df694a00f8a8437cb1d2d2d01a107afc [id=b54a76b761cc4db6b9926451b0fa1db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.680191 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e23759c43c49ba85a3ba8c47131850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.639086 2932 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a8dbc1165ac4f5b8ed792f6064cce6f with delete_type TABLET_DATA_DELETED (replaced by tablet b9c86c7d874a49178c7b2cd7316afa14) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:15.660187 2933 tablet_service.cc:1467] Processing CreateTablet for tablet 216b4311431b44bca04b6a681b35537b (DEFAULT_TABLE table=default.Table_a3208426010f46cf820de6a4dbba0099 [id=7efc17a625c6488e960a30fea44f1f0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.688987 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7dd7ce63f35444c83184b94e4898fdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.689635 2933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 216b4311431b44bca04b6a681b35537b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.700343 3965 leader_election.cc:288] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:15.701217 2649 tablet_service.cc:1467] Processing CreateTablet for tablet 46e23759c43c49ba85a3ba8c47131850 (DEFAULT_TABLE table=default.Table_df694a00f8a8437cb1d2d2d01a107afc [id=b54a76b761cc4db6b9926451b0fa1db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.701738 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:15.702414 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e23759c43c49ba85a3ba8c47131850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.703994 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:15.623970 3072 raft_consensus.cc:348] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.675796 2936 tablet_service.cc:1467] Processing CreateTablet for tablet 46e23759c43c49ba85a3ba8c47131850 (DEFAULT_TABLE table=default.Table_df694a00f8a8437cb1d2d2d01a107afc [id=b54a76b761cc4db6b9926451b0fa1db0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.705971 2936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 46e23759c43c49ba85a3ba8c47131850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.708416 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbeeccc19a504e7984b293127766daf0" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:15.716423 3072 raft_consensus.cc:374] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:15.716780 3072 raft_consensus.cc:729] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:15.717446 3072 consensus_queue.cc:260] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.676896 2943 tablet_service.cc:1467] Processing CreateTablet for tablet 4136d27807db40649d915422d7c10f6d (DEFAULT_TABLE table=default.Table_11f58699a34140ff8ad27a712df72ab2 [id=bd2a9767c4904134a96dc5c2a75159aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.719061 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4136d27807db40649d915422d7c10f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.719677 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af]). Expected version 2 got 0
I20260927 23:28:15.720450 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
W20260927 23:28:15.722434 2609 leader_election.cc:341] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:15.637548 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 90874.4us
I20260927 23:28:15.643831 2644 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a8dbc1165ac4f5b8ed792f6064cce6f with delete_type TABLET_DATA_DELETED (replaced by tablet b9c86c7d874a49178c7b2cd7316afa14) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:15.664516 3748 raft_consensus.cc:482] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:15.726735 3748 raft_consensus.cc:504] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.729034 3748 leader_election.cc:288] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:15.729529 3748 raft_consensus.cc:482] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:15.729934 3748 raft_consensus.cc:504] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.731881 3748 leader_election.cc:288] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:15.665100 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 216b4311431b44bca04b6a681b35537b (DEFAULT_TABLE table=default.Table_a3208426010f46cf820de6a4dbba0099 [id=7efc17a625c6488e960a30fea44f1f0e]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:15.734145 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 3a8dbc1165ac4f5b8ed792f6064cce6f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a8dbc1165ac4f5b8ed792f6064cce6f already in progress: creating tablet
I20260927 23:28:15.734787 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 216b4311431b44bca04b6a681b35537b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.734455 3942 tablet.cc:1722] T 1912a82ff46f4f9aaa24b7621cc3995d P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:15.670532 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 4136d27807db40649d915422d7c10f6d (DEFAULT_TABLE table=default.Table_11f58699a34140ff8ad27a712df72ab2 [id=bd2a9767c4904134a96dc5c2a75159aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.687964 3070 tablet_bootstrap.cc:492] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:15.736074 3070 ts_tablet_manager.cc:1397] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.265s user 0.062s sys 0.063s
W20260927 23:28:15.737538 2393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet b9c86c7d874a49178c7b2cd7316afa14 has no consensus state
I20260927 23:28:15.738024 2393 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:51972 (request call id 292) took 1131 ms. Trace:
I20260927 23:28:15.738379 3070 raft_consensus.cc:348] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.739003 3070 raft_consensus.cc:374] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:15.739236 3070 raft_consensus.cc:729] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:15.738291 2393 rpcz_store.cc:276] 0927 23:28:14.606314 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:15.640865 (+1034551us) service_pool.cc:227] Handling call
0927 23:28:15.641847 (+ 982us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:15.642510 (+ 663us) catalog_manager.cc:3616] Apply alter schema
0927 23:28:15.643104 (+ 594us) catalog_manager.cc:3639] Acquired catalog manager lock
0927 23:28:15.643307 (+ 203us) catalog_manager.cc:3786] Updating metadata on disk
0927 23:28:15.736925 (+ 93618us) catalog_manager.cc:3817] Committing alterations to in-memory state
0927 23:28:15.737956 (+ 1031us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:15.654075 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:15.654448 (+ 373us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:15.654472 (+ 24us) write_op.cc:435] Acquired schema lock
0927 23:28:15.654483 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:15.654762 (+ 279us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:15.654808 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:15.654830 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:15.654988 (+ 158us) tablet.cc:699] Row locks acquired
0927 23:28:15.655001 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:28:15.655118 (+ 117us) write_op.cc:270] Start()
0927 23:28:15.655203 (+ 85us) write_op.cc:276] Timestamp: P: 1790551695655047 usec, L: 0
0927 23:28:15.655218 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:15.655801 (+ 583us) log.cc:842] Serialized 405 byte log entry
0927 23:28:15.733556 (+ 77755us) op_driver.cc:444] REPLICATION: finished
0927 23:28:15.735087 (+ 1531us) write_op.cc:301] APPLY: starting
0927 23:28:15.735151 (+ 64us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:15.735252 (+ 101us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:15.735340 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:15.735490 (+ 150us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:15.735619 (+ 129us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:15.735658 (+ 39us) write_op.cc:312] APPLY: finished
0927 23:28:15.735890 (+ 232us) log.cc:842] Serialized 23 byte log entry
0927 23:28:15.736037 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:15.736109 (+ 72us) write_op.cc:454] Released schema lock
0927 23:28:15.736142 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:15.736356 (+ 214us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:15.736379 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1251,"apply.run_cpu_time_us":1468,"apply.run_wall_time_us":1468,"num_ops":1,"prepare.queue_time_us":6113,"prepare.run_cpu_time_us":2066,"prepare.run_wall_time_us":2066,"raft.queue_time_us":3188,"raft.run_cpu_time_us":747,"raft.run_wall_time_us":749,"replication_time_us":78200}]]}
I20260927 23:28:15.739914 3070 consensus_queue.cc:260] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.743227 3070 ts_tablet_manager.cc:1428] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260927 23:28:15.744216 3070 tablet_bootstrap.cc:492] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:15.749080 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4136d27807db40649d915422d7c10f6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.750530 3968 tablet.cc:1722] T 1912a82ff46f4f9aaa24b7621cc3995d P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:15.754092 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50dd9944d3134ab9b23a6109fd7d6ee5" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:15.756875 2644 tablet_service.cc:1467] Processing CreateTablet for tablet 82ff8173836e4cf2b43bf7e816a26f0a (DEFAULT_TABLE table=default.Table_b397e8b13f8e47a6a305e9f36d45f986 [id=35fe7c9553e443578f5eeff336bd55dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.759099 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0d8a1f277242588f57e4df95df5d5a" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:15.762857 2377 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9c86c7d874a49178c7b2cd7316afa14 (table default.Table_f45431c208e642ef9f406af42ff7d455 [id=d44851dff14d4e5084ae024995202140], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9c86c7d874a49178c7b2cd7316afa14 has no consensus state
I20260927 23:28:15.763589 2644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82ff8173836e4cf2b43bf7e816a26f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.765607 2802 tablet_service.cc:1467] Processing CreateTablet for tablet 82ff8173836e4cf2b43bf7e816a26f0a (DEFAULT_TABLE table=default.Table_b397e8b13f8e47a6a305e9f36d45f986 [id=35fe7c9553e443578f5eeff336bd55dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.756875 2653 tablet_service.cc:1467] Processing CreateTablet for tablet f102a35faef94cfd880f5b41d3e610e7 (DEFAULT_TABLE table=default.Table_b349caab5cc84842819f9a9bbfece301 [id=8ab43e9191d14f76ac0c8f79f39c9796]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:15.769119 2609 leader_election.cc:341] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:15.769531 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f102a35faef94cfd880f5b41d3e610e7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:15.769761 2609 leader_election.cc:341] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:15.771577 2907 leader_election.cc:341] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Not found: Tablet not found: 3c6e596517324300be30df8506a18904
I20260927 23:28:15.772969 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50dd9944d3134ab9b23a6109fd7d6ee5" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:15.774003 2963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0d8a1f277242588f57e4df95df5d5a" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:15.775367 2932 tablet_service.cc:1467] Processing CreateTablet for tablet 82ff8173836e4cf2b43bf7e816a26f0a (DEFAULT_TABLE table=default.Table_b397e8b13f8e47a6a305e9f36d45f986 [id=35fe7c9553e443578f5eeff336bd55dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.776641 2932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82ff8173836e4cf2b43bf7e816a26f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.772006 2907 leader_election.cc:302] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:15.779335 2387 catalog_manager.cc:5495] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } } }
I20260927 23:28:15.780887 2387 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af]). Expected version 2 got 0
I20260927 23:28:15.781682 2387 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:15.782312 2387 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
W20260927 23:28:15.783452 2608 leader_election.cc:341] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: bbeeccc19a504e7984b293127766daf0
I20260927 23:28:15.783943 2608 leader_election.cc:302] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
W20260927 23:28:15.785082 2608 leader_election.cc:341] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:15.785437 2608 leader_election.cc:302] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:15.786089 3965 raft_consensus.cc:2726] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:15.786398 2608 leader_election.cc:341] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:15.786832 3965 raft_consensus.cc:2726] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:15.786792 2608 leader_election.cc:302] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:15.787917 3965 raft_consensus.cc:2726] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:15.792690 2647 tablet_service.cc:1467] Processing CreateTablet for tablet f6a9b735499341a8857be0541a4deb24 (DEFAULT_TABLE table=default.Table_de478761aba74f9bab295c750c11178d [id=d492300842774a11ae7e4c86cd3438db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.793224 3943 raft_consensus.cc:482] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:15.793751 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 82ff8173836e4cf2b43bf7e816a26f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.793669 3943 raft_consensus.cc:504] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.794309 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a8dbc1165ac4f5b8ed792f6064cce6f with delete_type TABLET_DATA_DELETED (replaced by tablet b9c86c7d874a49178c7b2cd7316afa14) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:15.801304 2795 tablet_service.cc:1467] Processing CreateTablet for tablet f6a9b735499341a8857be0541a4deb24 (DEFAULT_TABLE table=default.Table_de478761aba74f9bab295c750c11178d [id=d492300842774a11ae7e4c86cd3438db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.801699 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6a9b735499341a8857be0541a4deb24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.802416 2795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6a9b735499341a8857be0541a4deb24. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:15.812842 2907 raft_consensus.cc:2670] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260927 23:28:15.813843 2804 tablet_service.cc:1467] Processing CreateTablet for tablet f102a35faef94cfd880f5b41d3e610e7 (DEFAULT_TABLE table=default.Table_b349caab5cc84842819f9a9bbfece301 [id=8ab43e9191d14f76ac0c8f79f39c9796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.820526 2387 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 1 got 0
I20260927 23:28:15.820544 3072 ts_tablet_manager.cc:1428] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.214s user 0.007s sys 0.000s
I20260927 23:28:15.823235 2387 catalog_manager.cc:5495] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: HEALTHY } } }
I20260927 23:28:15.824826 2387 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:15.833392 2937 tablet_service.cc:1467] Processing CreateTablet for tablet f102a35faef94cfd880f5b41d3e610e7 (DEFAULT_TABLE table=default.Table_b349caab5cc84842819f9a9bbfece301 [id=8ab43e9191d14f76ac0c8f79f39c9796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.834468 2946 tablet_service.cc:1467] Processing CreateTablet for tablet f6a9b735499341a8857be0541a4deb24 (DEFAULT_TABLE table=default.Table_de478761aba74f9bab295c750c11178d [id=d492300842774a11ae7e4c86cd3438db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.834779 3572 consensus_queue.cc:1035] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:15.837081 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6a9b735499341a8857be0541a4deb24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.840597 2937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f102a35faef94cfd880f5b41d3e610e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.821636 3167 tablet_bootstrap.cc:492] T 9eb27269a590476f878ee52a2aefd141 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:15.844884 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f102a35faef94cfd880f5b41d3e610e7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:15.845012 2906 consensus_peers.cc:487] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:15.845443 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 64509.8us
I20260927 23:28:15.846927 3866 raft_consensus.cc:482] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:15.847335 3866 raft_consensus.cc:504] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.848189 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe25672f4c34fc5ad5e5af3bd56bde9 (DEFAULT_TABLE table=default.Table_de44500abb18471d93e90c264b5491d5 [id=73ae2473e6a94d6780c75c3ad4a0c442]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.849279 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe25672f4c34fc5ad5e5af3bd56bde9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.849668 3070 tablet_bootstrap.cc:654] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:15.859637 3866 leader_election.cc:288] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:15.880681 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0578b6c99c4209a8d3c76f2089ef46" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:15.882370 3943 leader_election.cc:288] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:15.883255 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56766ed12a7c4a668a7e08fc1ad39d2f" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:15.884076 2672 raft_consensus.cc:2379] T 56766ed12a7c4a668a7e08fc1ad39d2f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate de216a4a8e4b4e1b857d8647a3b05f6d in this term.
I20260927 23:28:15.885553 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56766ed12a7c4a668a7e08fc1ad39d2f" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:28:15.886050 2906 leader_election.cc:341] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:15.887110 2761 leader_election.cc:302] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:15.887975 3943 raft_consensus.cc:3037] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:15.887797 2934 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe25672f4c34fc5ad5e5af3bd56bde9 (DEFAULT_TABLE table=default.Table_de44500abb18471d93e90c264b5491d5 [id=73ae2473e6a94d6780c75c3ad4a0c442]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.896410 2934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe25672f4c34fc5ad5e5af3bd56bde9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.897672 3943 raft_consensus.cc:2726] T 56766ed12a7c4a668a7e08fc1ad39d2f P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:15.906992 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 9fe25672f4c34fc5ad5e5af3bd56bde9 (DEFAULT_TABLE table=default.Table_de44500abb18471d93e90c264b5491d5 [id=73ae2473e6a94d6780c75c3ad4a0c442]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.907996 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9fe25672f4c34fc5ad5e5af3bd56bde9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.912137 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0578b6c99c4209a8d3c76f2089ef46" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:15.911808 2387 catalog_manager.cc:5495] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: HEALTHY } } }
I20260927 23:28:15.912904 2672 raft_consensus.cc:2454] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:15.913434 2387 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
W20260927 23:28:15.914016 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 47430.8us
I20260927 23:28:15.922254 2907 leader_election.cc:302] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:15.922605 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 584351ee27094b5192470975b95a2100 (DEFAULT_TABLE table=default.Table_6e2ee819c0404d22b95ad103ebf203e1 [id=80b78da80f7d484497476020d5e5bc13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.923121 3866 raft_consensus.cc:2781] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:15.923511 3866 raft_consensus.cc:482] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:15.923831 3866 raft_consensus.cc:3037] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:15.928723 2938 tablet_service.cc:1467] Processing CreateTablet for tablet 584351ee27094b5192470975b95a2100 (DEFAULT_TABLE table=default.Table_6e2ee819c0404d22b95ad103ebf203e1 [id=80b78da80f7d484497476020d5e5bc13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.929979 2938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 584351ee27094b5192470975b95a2100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.929718 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa0be1c90e047e0a83c97bebfb2121f (DEFAULT_TABLE table=default.Table_c79cb0e575124bc29051aa5905a314e9 [id=bc74da1c228a4926b7dbf5ecb28b3b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.931666 3572 raft_consensus.cc:482] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:15.932116 3572 raft_consensus.cc:504] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.934243 3572 leader_election.cc:288] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:15.935168 3866 raft_consensus.cc:504] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:15.939659 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb27269a590476f878ee52a2aefd141" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:15.941172 2907 leader_election.cc:341] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:15.950366 2796 tablet_service.cc:1467] Processing CreateTablet for tablet 584351ee27094b5192470975b95a2100 (DEFAULT_TABLE table=default.Table_6e2ee819c0404d22b95ad103ebf203e1 [id=80b78da80f7d484497476020d5e5bc13]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:15.951659 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 584351ee27094b5192470975b95a2100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.958400 3072 tablet_bootstrap.cc:492] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:15.951408 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb27269a590476f878ee52a2aefd141" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:15.965929 2906 leader_election.cc:341] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:15.972805 2906 leader_election.cc:302] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:15.973924 3572 raft_consensus.cc:2726] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:15.974462 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 584351ee27094b5192470975b95a2100. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.978627 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa0be1c90e047e0a83c97bebfb2121f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:15.983439 3070 tablet_bootstrap.cc:492] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:15.983963 3070 ts_tablet_manager.cc:1397] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.240s user 0.010s sys 0.000s
I20260927 23:28:15.986388 3070 raft_consensus.cc:348] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:15.998731 2282 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 51857.6us
I20260927 23:28:16.018457 3167 tablet_bootstrap.cc:654] T 9eb27269a590476f878ee52a2aefd141 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:16.038153 2392 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:16.039091 2392 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:16.040444 2392 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af]). Expected version 2 got 0
I20260927 23:28:16.041356 2392 catalog_manager.cc:5495] T a708a517b744496c83a3aa759b3f1081 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:16.042985 2392 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:16.055572 3866 leader_election.cc:288] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:16.046309 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa0be1c90e047e0a83c97bebfb2121f (DEFAULT_TABLE table=default.Table_c79cb0e575124bc29051aa5905a314e9 [id=bc74da1c228a4926b7dbf5ecb28b3b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.049213 2807 tablet_service.cc:1467] Processing CreateTablet for tablet c24145819c6f4312b9d1dc4a63ee66fd (DEFAULT_TABLE table=default.Table_86920ea4da9d4b3d85806280ca93e19e [id=39aaa13a28df4942b4e0600976c748aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.056109 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0578b6c99c4209a8d3c76f2089ef46" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:16.061522 2906 leader_election.cc:341] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:16.061762 3072 tablet_bootstrap.cc:654] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:16.067410 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c24145819c6f4312b9d1dc4a63ee66fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.071249 3336 ts_tablet_manager.cc:1918] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:16.081034 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa0be1c90e047e0a83c97bebfb2121f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.083259 2802 tablet_service.cc:1467] Processing CreateTablet for tablet dbd26104630049fba13ac57742a7ac56 (DEFAULT_TABLE table=default.Table_60becdf63fbc49b9a338f109290d00c8 [id=c7ed9dfe468c48248ab1df085d75b57e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.084789 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd26104630049fba13ac57742a7ac56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.092459 3070 raft_consensus.cc:374] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:16.092921 3070 raft_consensus.cc:729] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:16.093642 3070 consensus_queue.cc:260] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.094733 2946 tablet_service.cc:1467] Processing CreateTablet for tablet c24145819c6f4312b9d1dc4a63ee66fd (DEFAULT_TABLE table=default.Table_86920ea4da9d4b3d85806280ca93e19e [id=39aaa13a28df4942b4e0600976c748aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.095829 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c24145819c6f4312b9d1dc4a63ee66fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.056612 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0578b6c99c4209a8d3c76f2089ef46" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:16.085369 2647 tablet_service.cc:1467] Processing CreateTablet for tablet e47f72e2469a4e1b8a0b5d1821e2c028 (DEFAULT_TABLE table=default.Table_fdb186a310414586b0e653d8ef8dfeb0 [id=e6737d3426c54bbb85084bd35971a922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.086421 2653 tablet_service.cc:1467] Processing CreateTablet for tablet dbd26104630049fba13ac57742a7ac56 (DEFAULT_TABLE table=default.Table_60becdf63fbc49b9a338f109290d00c8 [id=c7ed9dfe468c48248ab1df085d75b57e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.050559 2650 tablet_service.cc:1467] Processing CreateTablet for tablet c24145819c6f4312b9d1dc4a63ee66fd (DEFAULT_TABLE table=default.Table_86920ea4da9d4b3d85806280ca93e19e [id=39aaa13a28df4942b4e0600976c748aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.098225 2947 tablet_service.cc:1467] Processing CreateTablet for tablet dbd26104630049fba13ac57742a7ac56 (DEFAULT_TABLE table=default.Table_60becdf63fbc49b9a338f109290d00c8 [id=c7ed9dfe468c48248ab1df085d75b57e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.099308 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd26104630049fba13ac57742a7ac56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.101161 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e47f72e2469a4e1b8a0b5d1821e2c028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.104785 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd26104630049fba13ac57742a7ac56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.105726 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c24145819c6f4312b9d1dc4a63ee66fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.097872 2674 raft_consensus.cc:3037] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:16.121378 2934 tablet_service.cc:1467] Processing CreateTablet for tablet 7fa0be1c90e047e0a83c97bebfb2121f (DEFAULT_TABLE table=default.Table_c79cb0e575124bc29051aa5905a314e9 [id=bc74da1c228a4926b7dbf5ecb28b3b72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.123859 2934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7fa0be1c90e047e0a83c97bebfb2121f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.135217 3866 consensus_queue.cc:1035] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:16.136792 2392 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_58948ac6bf854c31a26e8a6d47456c90 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 1 got 0
I20260927 23:28:16.138228 2392 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
I20260927 23:28:16.139865 2385 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:51972 (request call id 282) took 2585 ms. Trace:
I20260927 23:28:16.140156 2385 rpcz_store.cc:276] 0927 23:28:13.554020 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.899911 (+345891us) service_pool.cc:227] Handling call
0927 23:28:14.064687 (+164776us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b37d6 00007f8bb3bf9634 00007f8bb3bf95c2 00007f8bb3bf94f1 00007f8bb3bf946c 00007f8bb3bf8722 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:14.065828 (+ 1141us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:14.066833 (+ 1005us) catalog_manager.cc:3616] Apply alter schema
0927 23:28:14.067569 (+ 736us) catalog_manager.cc:3639] Acquired catalog manager lock
0927 23:28:14.067810 (+ 241us) catalog_manager.cc:3786] Updating metadata on disk
0927 23:28:14.352526 (+284716us) catalog_manager.cc:3817] Committing alterations to in-memory state
0927 23:28:16.139792 (+1787266us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:14.167101 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:14.167522 (+ 421us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:14.167550 (+ 28us) write_op.cc:435] Acquired schema lock
0927 23:28:14.167561 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:14.167837 (+ 276us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:14.167882 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:14.167903 (+ 21us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:14.168062 (+ 159us) tablet.cc:699] Row locks acquired
0927 23:28:14.168076 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:14.168189 (+ 113us) write_op.cc:270] Start()
0927 23:28:14.168273 (+ 84us) write_op.cc:276] Timestamp: P: 1790551694168113 usec, L: 0
0927 23:28:14.168288 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:14.301922 (+133634us) log.cc:842] Serialized 405 byte log entry
0927 23:28:14.302721 (+ 799us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068471 00007f8bb40df198 00007f8bb4540ade 00007f8bb45402b8 00007f8bb4544ba1 00007f8bb4544b59 00007f8bb4544ae9 00007f8bb4544ab1 00007f8bb4543dad 0000000000580784 00000000005805b9
0927 23:28:14.350182 (+ 47461us) op_driver.cc:444] REPLICATION: finished
0927 23:28:14.350503 (+ 321us) write_op.cc:301] APPLY: starting
0927 23:28:14.350568 (+ 65us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:14.350686 (+ 118us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:14.350707 (+ 21us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:14.350962 (+ 255us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:14.351123 (+ 161us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:14.351170 (+ 47us) write_op.cc:312] APPLY: finished
0927 23:28:14.351420 (+ 250us) log.cc:842] Serialized 23 byte log entry
0927 23:28:14.351597 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:14.351662 (+ 65us) write_op.cc:454] Released schema lock
0927 23:28:14.351695 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:14.351896 (+ 201us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:14.351919 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":376812416,"child_traces":[["op",{"apply.queue_time_us":102,"apply.run_cpu_time_us":1586,"apply.run_wall_time_us":1586,"num_ops":1,"prepare.queue_time_us":94,"prepare.run_cpu_time_us":3911,"prepare.run_wall_time_us":135805,"raft.queue_time_us":2474,"raft.run_cpu_time_us":759,"raft.run_wall_time_us":758,"replication_time_us":181781,"spinlock_wait_cycles":305979264}]]}
I20260927 23:28:16.161697 2802 tablet_service.cc:1467] Processing CreateTablet for tablet e47f72e2469a4e1b8a0b5d1821e2c028 (DEFAULT_TABLE table=default.Table_fdb186a310414586b0e653d8ef8dfeb0 [id=e6737d3426c54bbb85084bd35971a922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.162883 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e47f72e2469a4e1b8a0b5d1821e2c028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.162544 3975 tablet.cc:1722] T 1912a82ff46f4f9aaa24b7621cc3995d P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:16.196239 2820 raft_consensus.cc:3037] T 371842420a6048fca3cbc02d18f74ffc P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:16.205099 3621 consensus_queue.cc:1035] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:16.206749 2936 tablet_service.cc:1467] Processing CreateTablet for tablet e47f72e2469a4e1b8a0b5d1821e2c028 (DEFAULT_TABLE table=default.Table_fdb186a310414586b0e653d8ef8dfeb0 [id=e6737d3426c54bbb85084bd35971a922]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.213279 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:16.214351 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1912a82ff46f4f9aaa24b7621cc3995d (table default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458]). Expected version 1 got 0
I20260927 23:28:16.215204 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:16.215845 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af]). Expected version 2 got 0
I20260927 23:28:16.216375 2393 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1c87484b8b9742faa683f13d37308e58 (table default.Table_4fa6aec871b84978973cb579e195404b [id=2cd91cbdb5b14e16b847253971c8cb25]). Expected version 1 got 0
I20260927 23:28:16.224093 2644 tablet_service.cc:1467] Processing CreateTablet for tablet a2da2780fde74a3cae8e3ca53b662539 (DEFAULT_TABLE table=default.Table_8fc13e9bc9e644779d259edabc03ecb6 [id=9ae9f28460514548ab2ba8d266a653ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.225394 2644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2da2780fde74a3cae8e3ca53b662539. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.229266 2796 tablet_service.cc:1467] Processing CreateTablet for tablet a2da2780fde74a3cae8e3ca53b662539 (DEFAULT_TABLE table=default.Table_8fc13e9bc9e644779d259edabc03ecb6 [id=9ae9f28460514548ab2ba8d266a653ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.230543 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2da2780fde74a3cae8e3ca53b662539. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.233080 2943 tablet_service.cc:1467] Processing CreateTablet for tablet a2da2780fde74a3cae8e3ca53b662539 (DEFAULT_TABLE table=default.Table_8fc13e9bc9e644779d259edabc03ecb6 [id=9ae9f28460514548ab2ba8d266a653ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:16.248867 3167 tablet_bootstrap.cc:492] T 9eb27269a590476f878ee52a2aefd141 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:16.249352 3167 ts_tablet_manager.cc:1397] T 9eb27269a590476f878ee52a2aefd141 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.428s user 0.010s sys 0.003s
I20260927 23:28:16.264688 3572 raft_consensus.cc:482] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:16.265417 3572 raft_consensus.cc:504] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.272850 3070 ts_tablet_manager.cc:1428] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.289s user 0.006s sys 0.003s
I20260927 23:28:16.273885 3070 tablet_bootstrap.cc:492] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:16.279170 3072 tablet_bootstrap.cc:492] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:16.279672 3072 ts_tablet_manager.cc:1397] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.321s user 0.011s sys 0.005s
I20260927 23:28:16.284575 3070 tablet_bootstrap.cc:654] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:16.312783 2674 raft_consensus.cc:2454] T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:16.318948 2936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e47f72e2469a4e1b8a0b5d1821e2c028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.321130 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2da2780fde74a3cae8e3ca53b662539. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:16.323679 3167 raft_consensus.cc:348] T 9eb27269a590476f878ee52a2aefd141 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.324488 3167 raft_consensus.cc:374] T 9eb27269a590476f878ee52a2aefd141 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:16.324726 3167 raft_consensus.cc:729] T 9eb27269a590476f878ee52a2aefd141 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:16.335259 3558 ts_tablet_manager.cc:1918] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:16.336661 3558 log.cc:1196] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/702ebdeb7c5a4b7d900dbb00236cb8ae
I20260927 23:28:16.338944 3558 ts_tablet_manager.cc:1939] T 702ebdeb7c5a4b7d900dbb00236cb8ae P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:16.071662 3336 log.cc:1196] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/3c6e596517324300be30df8506a18904
I20260927 23:28:16.351245 3336 ts_tablet_manager.cc:1939] T 3c6e596517324300be30df8506a18904 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:16.356479 3963 raft_consensus.cc:482] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:16.357016 3963 raft_consensus.cc:504] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.359171 3963 leader_election.cc:288] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:16.361999 3070 tablet_bootstrap.cc:492] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:16.362430 3070 ts_tablet_manager.cc:1397] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.089s user 0.001s sys 0.009s
I20260927 23:28:16.367323 2385 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:16.379302 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 216) took 2851 ms. Trace:
I20260927 23:28:16.382038 3965 raft_consensus.cc:482] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:16.382520 3965 raft_consensus.cc:504] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.385100 3965 leader_election.cc:288] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:16.388005 3070 raft_consensus.cc:348] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.388835 3070 raft_consensus.cc:374] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:16.389078 3070 raft_consensus.cc:729] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:16.377110 3072 raft_consensus.cc:348] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.365931 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4a9d69d75c4bfbb43e558b83b460d1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:16.393322 2609 leader_election.cc:341] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:16.393535 2385 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1c87484b8b9742faa683f13d37308e58 (table default.Table_4fa6aec871b84978973cb579e195404b [id=2cd91cbdb5b14e16b847253971c8cb25]). Expected version 1 got 0
I20260927 23:28:16.354125 3167 consensus_queue.cc:260] T 9eb27269a590476f878ee52a2aefd141 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.389209 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c87484b8b9742faa683f13d37308e58" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:16.394809 2824 raft_consensus.cc:2454] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:16.395788 3167 ts_tablet_manager.cc:1428] T 9eb27269a590476f878ee52a2aefd141 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.146s user 0.007s sys 0.000s
I20260927 23:28:16.396091 3558 rpcz_store.cc:276] 0927 23:28:13.528006 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.528508 (+ 502us) service_pool.cc:227] Handling call
0927 23:28:15.599156 (+2070648us) ts_tablet_manager.cc:1145] Deleting tablet 702ebdeb7c5a4b7d900dbb00236cb8ae
0927 23:28:15.599209 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.001513 (+402304us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:16.334922 (+333409us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:16.379231 (+ 44309us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2069946}
I20260927 23:28:16.398315 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 217) took 2239 ms. Trace:
I20260927 23:28:16.398675 3558 rpcz_store.cc:276] 0927 23:28:14.158740 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:14.159217 (+ 477us) service_pool.cc:227] Handling call
0927 23:28:16.397973 (+2238756us) ts_tablet_manager.cc:1145] Deleting tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc
0927 23:28:16.398016 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.398262 (+ 246us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2238129}
I20260927 23:28:16.392086 3072 raft_consensus.cc:374] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:16.407387 2609 leader_election.cc:302] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:16.408370 3965 raft_consensus.cc:2781] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:16.408732 3965 raft_consensus.cc:482] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:16.409189 3965 raft_consensus.cc:3037] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:16.417482 3965 raft_consensus.cc:504] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.400118 2907 leader_election.cc:302] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:16.400755 3072 raft_consensus.cc:729] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
W20260927 23:28:16.413213 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 129765us
I20260927 23:28:16.427306 3558 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42492 (request call id 218) took 2268 ms. Trace:
I20260927 23:28:16.427583 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1890852164d492db3540a97847888fd" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:16.427657 3558 rpcz_store.cc:276] 0927 23:28:14.159088 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:14.159932 (+ 844us) service_pool.cc:227] Handling call
0927 23:28:16.426929 (+2266997us) ts_tablet_manager.cc:1145] Deleting tablet 0e0c12031a734ca0a6536404409c62ec
0927 23:28:16.426973 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.427247 (+ 274us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2266374}
I20260927 23:28:16.397459 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 702ebdeb7c5a4b7d900dbb00236cb8ae (table default.Table_f1571426f866470d94cfe827bd30ba79 [id=0bcc62622acc4fb7a60d137f1b75265d]) successfully deleted
W20260927 23:28:16.429495 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc already in progress: creating tablet
W20260927 23:28:16.430311 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 0e0c12031a734ca0a6536404409c62ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e0c12031a734ca0a6536404409c62ec already in progress: creating tablet
W20260927 23:28:16.430922 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 78e099028f7847d1b3de7d437be97de0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78e099028f7847d1b3de7d437be97de0 already in progress: creating tablet
W20260927 23:28:16.431526 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet e81df7c7f1a04eb9904dd2a1a5a596c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e81df7c7f1a04eb9904dd2a1a5a596c4 already in progress: creating tablet
W20260927 23:28:16.432088 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 3a8dbc1165ac4f5b8ed792f6064cce6f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a8dbc1165ac4f5b8ed792f6064cce6f already in progress: creating tablet
I20260927 23:28:16.436785 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c87484b8b9742faa683f13d37308e58" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:16.437381 2824 raft_consensus.cc:3037] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:16.441098 3572 leader_election.cc:288] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:16.441638 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 187) took 5726 ms. Trace:
I20260927 23:28:16.441953 3336 rpcz_store.cc:276] 0927 23:28:10.743593 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:10.744976 (+ 1383us) service_pool.cc:227] Handling call
0927 23:28:15.552950 (+4807974us) ts_tablet_manager.cc:1145] Deleting tablet 3c6e596517324300be30df8506a18904
0927 23:28:15.553006 (+ 56us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:15.985507 (+432501us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:16.070989 (+ 85482us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:16.441574 (+370585us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13595,"tablet-delete.queue_time_us":4777227}
I20260927 23:28:16.443620 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 191) took 5411 ms. Trace:
I20260927 23:28:16.445494 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 3c6e596517324300be30df8506a18904 (table default.Table_585d1f792a1b42e7ae86d5ecb5edaff2 [id=ff09406f8316449ead4cffa47a2095b8]) successfully deleted
I20260927 23:28:16.443928 3336 rpcz_store.cc:276] 0927 23:28:11.031998 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:11.033545 (+ 1547us) service_pool.cc:227] Handling call
0927 23:28:16.443165 (+5409620us) ts_tablet_manager.cc:1145] Deleting tablet c848ed8a221a4b799529ccc0d1830893
0927 23:28:16.443230 (+ 65us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.443550 (+ 320us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":5409090}
W20260927 23:28:16.451004 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet c848ed8a221a4b799529ccc0d1830893 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c848ed8a221a4b799529ccc0d1830893 already in progress: creating tablet
I20260927 23:28:16.424356 3072 consensus_queue.cc:260] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.452708 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 198) took 4493 ms. Trace:
I20260927 23:28:16.453121 3336 rpcz_store.cc:276] 0927 23:28:11.959704 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:11.959868 (+ 164us) service_pool.cc:227] Handling call
0927 23:28:16.452139 (+4492271us) ts_tablet_manager.cc:1145] Deleting tablet 761a67c27c7a49a3903ed6f2bf5b96d3
0927 23:28:16.452200 (+ 61us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.452645 (+ 445us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4491638}
W20260927 23:28:16.454412 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 761a67c27c7a49a3903ed6f2bf5b96d3 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 761a67c27c7a49a3903ed6f2bf5b96d3 already in progress: creating tablet
I20260927 23:28:16.455699 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 200) took 4327 ms. Trace:
I20260927 23:28:16.456176 3336 rpcz_store.cc:276] 0927 23:28:12.127923 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.133763 (+ 5840us) service_pool.cc:227] Handling call
0927 23:28:16.455228 (+4321465us) ts_tablet_manager.cc:1145] Deleting tablet bc9fb7e66b6b45c29688d9379e24d95d
0927 23:28:16.455301 (+ 73us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.455626 (+ 325us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4320234}
W20260927 23:28:16.456703 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 56682.6us
I20260927 23:28:16.460455 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4a9d69d75c4bfbb43e558b83b460d1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:16.460572 3621 consensus_queue.cc:1035] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:16.461074 2968 raft_consensus.cc:2454] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:16.461254 3968 tablet.cc:1722] T 7f530db3b52944f6a283ee9e2bf66828 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:16.457049 3866 raft_consensus.cc:2781] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:16.462945 3866 raft_consensus.cc:686] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:16.463704 3866 consensus_queue.cc:237] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.465999 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c87484b8b9742faa683f13d37308e58" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:16.466562 2968 raft_consensus.cc:2454] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:16.468963 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1890852164d492db3540a97847888fd" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:16.469697 2674 raft_consensus.cc:2454] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
W20260927 23:28:16.471640 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet bc9fb7e66b6b45c29688d9379e24d95d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bc9fb7e66b6b45c29688d9379e24d95d already in progress: creating tablet
I20260927 23:28:16.472069 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 201) took 4336 ms. Trace:
I20260927 23:28:16.473115 2824 raft_consensus.cc:2454] T 1c87484b8b9742faa683f13d37308e58 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:16.473999 3336 rpcz_store.cc:276] 0927 23:28:12.135689 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.135795 (+ 106us) service_pool.cc:227] Handling call
0927 23:28:16.470962 (+4335167us) ts_tablet_manager.cc:1145] Deleting tablet 7a5274b56b814c029e02c16243a3e663
0927 23:28:16.471686 (+ 724us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.472000 (+ 314us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4334589}
I20260927 23:28:16.481303 2609 leader_election.cc:302] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:16.482712 3963 raft_consensus.cc:2781] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:16.483237 3963 raft_consensus.cc:686] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:16.484495 2385 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
I20260927 23:28:16.484139 3963 consensus_queue.cc:237] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.428086 3070 consensus_queue.cc:260] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.489446 3070 ts_tablet_manager.cc:1428] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.127s user 0.006s sys 0.000s
I20260927 23:28:16.418159 3167 tablet_bootstrap.cc:492] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:16.475871 2907 leader_election.cc:302] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:16.499586 3965 leader_election.cc:288] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:16.479763 3017 heartbeater.cc:347] Connected to a master server at 127.2.57.60:41571
I20260927 23:28:16.483767 3072 ts_tablet_manager.cc:1428] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.204s user 0.007s sys 0.003s
W20260927 23:28:16.501278 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 7a5274b56b814c029e02c16243a3e663 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a5274b56b814c029e02c16243a3e663 already in progress: creating tablet
I20260927 23:28:16.506040 2385 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af]). Expected version 2 got 0
I20260927 23:28:16.507207 2385 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_55dca2c970f045b6966759363c4a0816 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 2 got 0
I20260927 23:28:16.507725 2385 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:16.508630 2385 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 6d0d8a1f277242588f57e4df95df5d5a (table default.Table_47fc9fc598ce4ca4bb4f94a523b39f74 [id=7b1bf5063eef401eb9de9b6f7aa40e4b]). Expected version 1 got 0
I20260927 23:28:16.511694 3981 raft_consensus.cc:2781] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:16.512071 3981 raft_consensus.cc:482] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:16.512426 3981 raft_consensus.cc:3037] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:16.500198 2906 leader_election.cc:341] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:16.522687 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 44762.8us
I20260927 23:28:16.528389 2608 leader_election.cc:302] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:16.533613 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c87484b8b9742faa683f13d37308e58" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:16.533788 3928 raft_consensus.cc:2781] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:16.534161 3928 raft_consensus.cc:482] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:16.534263 2968 raft_consensus.cc:3037] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:16.534467 3928 raft_consensus.cc:3037] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:16.537781 3982 consensus_queue.cc:1035] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:16.542395 3928 raft_consensus.cc:504] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.559211 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4a9d69d75c4bfbb43e558b83b460d1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
W20260927 23:28:16.560741 2609 leader_election.cc:341] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:16.562621 3998 tablet.cc:1722] T 7f530db3b52944f6a283ee9e2bf66828 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:16.563588 3965 raft_consensus.cc:482] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:16.564146 3965 raft_consensus.cc:504] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.573976 2966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4a9d69d75c4bfbb43e558b83b460d1" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:16.574584 2966 raft_consensus.cc:3037] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:16.394546 3991 tablet.cc:1722] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:16.584980 3981 raft_consensus.cc:504] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.597340 3070 tablet_bootstrap.cc:492] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:16.601210 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 202) took 4445 ms. Trace:
I20260927 23:28:16.589838 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1890852164d492db3540a97847888fd" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:16.601643 3336 rpcz_store.cc:276] 0927 23:28:12.155739 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.155898 (+ 159us) service_pool.cc:227] Handling call
0927 23:28:16.501797 (+4345899us) ts_tablet_manager.cc:1145] Deleting tablet 972763afd57847269fafadbdd2a5f3d6
0927 23:28:16.520719 (+ 18922us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.601125 (+ 80406us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":4345242}
I20260927 23:28:16.599452 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f253a54ab3b44822aee266314fa45495" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:16.603246 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 972763afd57847269fafadbdd2a5f3d6 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 972763afd57847269fafadbdd2a5f3d6 already in progress: creating tablet
I20260927 23:28:16.605768 3072 tablet_bootstrap.cc:492] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
W20260927 23:28:16.606869 2906 leader_election.cc:341] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:16.619714 3986 log.cc:924] Time spent T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: Append to log took a long time: real 0.264s user 0.029s sys 0.001s
I20260927 23:28:16.620414 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 209) took 3972 ms. Trace:
I20260927 23:28:16.620877 3336 rpcz_store.cc:276] 0927 23:28:12.648242 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.648437 (+ 195us) service_pool.cc:227] Handling call
0927 23:28:16.608053 (+3959616us) ts_tablet_manager.cc:1145] Deleting tablet fc60d0c7a3794607814393f033238052
0927 23:28:16.610400 (+ 2347us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.620336 (+ 9936us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3953299}
W20260927 23:28:16.622606 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet fc60d0c7a3794607814393f033238052 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet fc60d0c7a3794607814393f033238052 already in progress: creating tablet
I20260927 23:28:16.623943 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 213) took 3643 ms. Trace:
I20260927 23:28:16.624353 3336 rpcz_store.cc:276] 0927 23:28:12.980975 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.984392 (+ 3417us) service_pool.cc:227] Handling call
0927 23:28:16.623531 (+3639139us) ts_tablet_manager.cc:1145] Deleting tablet 10fe6b0a4c614b77ae95b200b6a12843
0927 23:28:16.623584 (+ 53us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.623886 (+ 302us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3638420}
W20260927 23:28:16.625720 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 10fe6b0a4c614b77ae95b200b6a12843 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 10fe6b0a4c614b77ae95b200b6a12843 already in progress: creating tablet
I20260927 23:28:16.626915 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 216) took 3394 ms. Trace:
I20260927 23:28:16.627272 3336 rpcz_store.cc:276] 0927 23:28:13.232821 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.232938 (+ 117us) service_pool.cc:227] Handling call
0927 23:28:16.626522 (+3393584us) ts_tablet_manager.cc:1145] Deleting tablet 9af9b87abf1f4f78b3140a91f2525066
0927 23:28:16.626572 (+ 50us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.626861 (+ 289us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3392988}
I20260927 23:28:16.628592 3167 tablet_bootstrap.cc:700] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51: Time spent opening tablet: real 0.133s user 0.004s sys 0.000s
W20260927 23:28:16.629163 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 9af9b87abf1f4f78b3140a91f2525066 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9af9b87abf1f4f78b3140a91f2525066 already in progress: creating tablet
I20260927 23:28:16.630302 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 219) took 3068 ms. Trace:
I20260927 23:28:16.602545 3981 leader_election.cc:288] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:16.633054 3167 tablet_bootstrap.cc:654] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:16.633800 3336 rpcz_store.cc:276] 0927 23:28:13.561793 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.566552 (+ 4759us) service_pool.cc:227] Handling call
0927 23:28:16.630018 (+3063466us) ts_tablet_manager.cc:1145] Deleting tablet 702ebdeb7c5a4b7d900dbb00236cb8ae
0927 23:28:16.630062 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.630250 (+ 188us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3061918}
I20260927 23:28:16.594887 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1890852164d492db3540a97847888fd" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:16.635707 2669 raft_consensus.cc:3037] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:16.568053 3928 leader_election.cc:288] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:16.603906 2609 leader_election.cc:341] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:16.653187 2669 raft_consensus.cc:2454] T a1890852164d492db3540a97847888fd P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:16.654927 3965 leader_election.cc:288] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:16.635172 3996 log.cc:924] Time spent T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1: Append to log took a long time: real 0.124s user 0.001s sys 0.000s
W20260927 23:28:16.677991 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae because the tablet was not found. No further retry: Not found: Tablet not found: 702ebdeb7c5a4b7d900dbb00236cb8ae
I20260927 23:28:16.681015 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 221) took 3118 ms. Trace:
I20260927 23:28:16.659608 2907 leader_election.cc:302] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:16.682397 3981 raft_consensus.cc:2781] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:16.682937 3981 raft_consensus.cc:686] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:16.683787 3981 consensus_queue.cc:237] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.608793 2968 raft_consensus.cc:2454] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:16.654726 2966 raft_consensus.cc:2454] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:16.693744 2608 leader_election.cc:302] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:16.701069 4001 raft_consensus.cc:2781] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:16.701511 3167 tablet_bootstrap.cc:492] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:16.701895 4001 raft_consensus.cc:686] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:16.702728 4001 consensus_queue.cc:237] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.706403 2966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f253a54ab3b44822aee266314fa45495" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:16.707365 3336 rpcz_store.cc:276] 0927 23:28:13.562631 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.568401 (+ 5770us) service_pool.cc:227] Handling call
0927 23:28:16.680676 (+3112275us) ts_tablet_manager.cc:1145] Deleting tablet 4929fc9e396e48a189c94586d58aa55f
0927 23:28:16.680728 (+ 52us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.680944 (+ 216us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3111438}
W20260927 23:28:16.708725 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 4929fc9e396e48a189c94586d58aa55f because the tablet was not found. No further retry: Not found: Tablet not found: 4929fc9e396e48a189c94586d58aa55f
I20260927 23:28:16.709272 3167 ts_tablet_manager.cc:1397] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.291s user 0.009s sys 0.002s
I20260927 23:28:16.709887 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 220) took 3147 ms. Trace:
I20260927 23:28:16.710206 3336 rpcz_store.cc:276] 0927 23:28:13.562236 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.569366 (+ 7130us) service_pool.cc:227] Handling call
0927 23:28:16.709583 (+3140217us) ts_tablet_manager.cc:1145] Deleting tablet cc1c0da6ea5c4a09a71555f37becae33
0927 23:28:16.709631 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.709826 (+ 195us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":3139558}
I20260927 23:28:16.715466 2386 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 1da38499af164d7fb5c1a8d65167ad24
I20260927 23:28:16.717631 2805 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da38499af164d7fb5c1a8d65167ad24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:14 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:16.718340 3558 tablet_replica.cc:330] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:16.719193 3558 raft_consensus.cc:2229] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:16.719709 3558 raft_consensus.cc:2258] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:16.721860 3558 ts_tablet_manager.cc:1905] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:16.727062 2389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 48) took 2056 ms (client timeout 1000 ms). Trace:
W20260927 23:28:16.727470 2389 rpcz_store.cc:269] 0927 23:28:14.670757 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:14.722901 (+ 52144us) service_pool.cc:227] Handling call
0927 23:28:16.726945 (+2004044us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2695,"reported_tablets":11}
I20260927 23:28:16.731873 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:16.732580 2805 tablet_service.cc:1514] Processing DeleteTablet for tablet 2411d7c53a154fafad9edda9fa75bb36 with delete_type TABLET_DATA_DELETED (replaced by tablet 73cfb41befe44e5189029abafe7e326d) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:16.735674 2653 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:16.738152 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0115095a82489a8aeb941efcc8a979 with delete_type TABLET_DATA_DELETED (replaced by tablet 178503861b1648d9a931d1606d807ed8) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:16.738777 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 2411d7c53a154fafad9edda9fa75bb36 with delete_type TABLET_DATA_DELETED (replaced by tablet 73cfb41befe44e5189029abafe7e326d) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:16.739601 3167 raft_consensus.cc:348] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.750488 2386 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 281) took 3421 ms. Trace:
I20260927 23:28:16.748117 3167 raft_consensus.cc:374] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:16.751010 3167 raft_consensus.cc:729] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:16.751034 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da38499af164d7fb5c1a8d65167ad24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:14 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:16.754058 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 because the tablet was not found. No further retry: Not found: Tablet not found: 5bf856a05dff4fb4943fe8e1d9b7f385
W20260927 23:28:16.754843 2384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 47) took 3114 ms (client timeout 1000 ms). Trace:
W20260927 23:28:16.755137 2384 rpcz_store.cc:269] 0927 23:28:13.640420 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.642513 (+ 2093us) service_pool.cc:227] Handling call
0927 23:28:16.754748 (+3112235us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":9}
I20260927 23:28:16.755311 3167 consensus_queue.cc:260] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:16.755792 2388 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 33) took 4013 ms (client timeout 1000 ms). Trace:
I20260927 23:28:16.756155 2805 tablet_service.cc:1514] Processing DeleteTablet for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae with delete_type TABLET_DATA_DELETED (replaced by tablet 8fadaafd49b04b9483a6a50f2f7d5be7) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:16.756773 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0115095a82489a8aeb941efcc8a979 with delete_type TABLET_DATA_DELETED (replaced by tablet 178503861b1648d9a931d1606d807ed8) from {username='slave'} at 127.0.0.1:42492
W20260927 23:28:16.763597 2390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 30) took 2718 ms (client timeout 1000 ms). Trace:
W20260927 23:28:16.765900 2393 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 49) took 1098 ms (client timeout 1000 ms). Trace:
W20260927 23:28:16.766347 2393 rpcz_store.cc:269] 0927 23:28:15.667414 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:15.743348 (+ 75934us) service_pool.cc:227] Handling call
0927 23:28:16.765829 (+1022481us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":14}
W20260927 23:28:16.771226 2392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 35) took 1901 ms (client timeout 1000 ms). Trace:
I20260927 23:28:16.773488 2647 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:16.757323 3167 ts_tablet_manager.cc:1428] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.048s user 0.005s sys 0.000s
I20260927 23:28:16.775395 3167 tablet_bootstrap.cc:492] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:16.782491 4003 tablet_replica.cc:330] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:16.785228 4003 raft_consensus.cc:2229] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:16.786931 4003 raft_consensus.cc:2258] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:16.773989 2391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 34) took 3026 ms (client timeout 1000 ms). Trace:
W20260927 23:28:16.781672 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet fd0115095a82489a8aeb941efcc8a979 because the tablet was not found. No further retry: Not found: Tablet not found: fd0115095a82489a8aeb941efcc8a979
W20260927 23:28:16.787588 2391 rpcz_store.cc:269] 0927 23:28:13.747723 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.756471 (+ 8748us) service_pool.cc:227] Handling call
0927 23:28:16.773907 (+3017436us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":287631,"reported_tablets":22}
I20260927 23:28:16.750813 2386 rpcz_store.cc:276] 0927 23:28:13.328812 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:13.897874 (+569062us) service_pool.cc:227] Handling call
0927 23:28:14.057616 (+159742us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b3466 00007f8bb3bf4b34 00007f8bb3bf4ac2 00007f8bb3bf49f1 00007f8bb3bf496c 00007f8bb3bf3c22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:14.059286 (+ 1670us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:14.060017 (+ 731us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:28:14.060274 (+ 257us) catalog_manager.cc:2761] Locking tablets
0927 23:28:14.061199 (+ 925us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:28:14.160359 (+ 99160us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:28:14.161736 (+ 1377us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:16.715325 (+2553589us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:16.750405 (+ 35080us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:14.095251 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:14.095763 (+ 512us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:14.095801 (+ 38us) write_op.cc:435] Acquired schema lock
0927 23:28:14.095824 (+ 23us) tablet.cc:660] Decoding operations
0927 23:28:14.096189 (+ 365us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:14.096253 (+ 64us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:14.096288 (+ 35us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:14.096682 (+ 394us) tablet.cc:699] Row locks acquired
0927 23:28:14.096709 (+ 27us) write_op.cc:260] PREPARE: finished
0927 23:28:14.096851 (+ 142us) write_op.cc:270] Start()
0927 23:28:14.097018 (+ 167us) write_op.cc:276] Timestamp: P: 1790551694096763 usec, L: 0
0927 23:28:14.097048 (+ 30us) op_driver.cc:331] REPLICATION: starting
0927 23:28:14.097959 (+ 911us) log.cc:842] Serialized 787 byte log entry
0927 23:28:14.124609 (+ 26650us) op_driver.cc:444] REPLICATION: finished
0927 23:28:14.127330 (+ 2721us) write_op.cc:301] APPLY: starting
0927 23:28:14.127382 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:14.127478 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:14.127565 (+ 87us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:14.127856 (+ 291us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:14.127985 (+ 129us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:14.128027 (+ 42us) write_op.cc:312] APPLY: finished
0927 23:28:14.128250 (+ 223us) log.cc:842] Serialized 29 byte log entry
0927 23:28:14.145018 (+ 16768us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:14.145088 (+ 70us) write_op.cc:454] Released schema lock
0927 23:28:14.145124 (+ 36us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:14.145333 (+ 209us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:14.145357 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":364211584,"child_traces":[["op",{"apply.queue_time_us":2581,"apply.run_cpu_time_us":1645,"apply.run_wall_time_us":18208,"num_ops":2,"prepare.queue_time_us":31258,"prepare.run_cpu_time_us":3063,"prepare.run_wall_time_us":3063,"replication_time_us":27483,"thread_start_us":175819,"threads_started":1}]]}
W20260927 23:28:16.756068 2388 rpcz_store.cc:269] 0927 23:28:12.723095 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:12.930498 (+207403us) service_pool.cc:227] Handling call
0927 23:28:13.045089 (+114591us) spinlock_profiling.cc:243] Waited 94.2 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:16.736247 (+3691158us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:13.340081 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:13.340515 (+ 434us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:13.340539 (+ 24us) write_op.cc:435] Acquired schema lock
0927 23:28:13.340550 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:13.340845 (+ 295us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:13.340883 (+ 38us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:13.340902 (+ 19us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:13.341043 (+ 141us) tablet.cc:699] Row locks acquired
0927 23:28:13.341055 (+ 12us) write_op.cc:260] PREPARE: finished
0927 23:28:13.341170 (+ 115us) write_op.cc:270] Start()
0927 23:28:13.341257 (+ 87us) write_op.cc:276] Timestamp: P: 1790551693341131 usec, L: 0
0927 23:28:13.341272 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:13.341816 (+ 544us) log.cc:842] Serialized 524 byte log entry
0927 23:28:13.374729 (+ 32913us) op_driver.cc:444] REPLICATION: finished
0927 23:28:13.380443 (+ 5714us) write_op.cc:301] APPLY: starting
0927 23:28:13.380494 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:13.380588 (+ 94us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:13.380669 (+ 81us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:13.380848 (+ 179us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:13.380976 (+ 128us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:13.381018 (+ 42us) write_op.cc:312] APPLY: finished
0927 23:28:13.381203 (+ 185us) log.cc:842] Serialized 23 byte log entry
0927 23:28:13.381324 (+ 121us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.381390 (+ 66us) write_op.cc:454] Released schema lock
0927 23:28:13.381415 (+ 25us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:13.381581 (+ 166us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.381604 (+ 23us) write_op.cc:454] Released schema lock
Related trace 'op':
0927 23:28:13.706620 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:13.707133 (+ 513us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:13.707161 (+ 28us) write_op.cc:435] Acquired schema lock
0927 23:28:13.707175 (+ 14us) tablet.cc:660] Decoding operations
0927 23:28:13.707493 (+ 318us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:13.707544 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:13.707570 (+ 26us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:13.707739 (+ 169us) tablet.cc:699] Row locks acquired
0927 23:28:13.707753 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:13.707882 (+ 129us) write_op.cc:270] Start()
0927 23:28:13.707975 (+ 93us) write_op.cc:276] Timestamp: P: 1790551693707801 usec, L: 0
0927 23:28:13.707992 (+ 17us) op_driver.cc:331] REPLICATION: starting
0927 23:28:13.726227 (+ 18235us) log.cc:842] Serialized 356 byte log entry
0927 23:28:13.745381 (+ 19154us) op_driver.cc:444] REPLICATION: finished
0927 23:28:13.750098 (+ 4717us) write_op.cc:301] APPLY: starting
0927 23:28:13.750158 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:13.750251 (+ 93us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:13.750339 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:13.750524 (+ 185us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:13.750648 (+ 124us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:13.750689 (+ 41us) write_op.cc:312] APPLY: finished
0927 23:28:13.750874 (+ 185us) log.cc:842] Serialized 23 byte log entry
0927 23:28:13.751029 (+ 155us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.751099 (+ 70us) write_op.cc:454] Released schema lock
0927 23:28:13.751131 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:13.751315 (+ 184us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:13.751347 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":19,"spinlock_wait_cycles":216655616,"child_traces":[["op",{"apply.queue_time_us":5572,"apply.run_cpu_time_us":1322,"apply.run_wall_time_us":1322,"num_ops":1,"prepare.queue_time_us":8143,"prepare.run_cpu_time_us":2099,"prepare.run_wall_time_us":92386,"raft.queue_time_us":95,"raft.run_cpu_time_us":716,"raft.run_wall_time_us":1707,"replication_time_us":33368,"thread_start_us":58618,"threads_started":1}],["op",{"apply.queue_time_us":4546,"apply.run_cpu_time_us":1404,"apply.run_wall_time_us":1404,"num_ops":1,"prepare.queue_time_us":449,"prepare.run_cpu_time_us":2287,"prepare.run_wall_time_us":21024,"raft.queue_time_us":5891,"raft.run_cpu_time_us":764,"raft.run_wall_time_us":764,"replication_time_us":37304}]]}
I20260927 23:28:16.792150 4003 ts_tablet_manager.cc:1905] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:16.763953 2390 rpcz_store.cc:269] 0927 23:28:14.045813 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:14.075490 (+ 29677us) service_pool.cc:227] Handling call
0927 23:28:16.763518 (+2688028us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":45258,"reported_tablets":16}
W20260927 23:28:16.796680 2387 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 31) took 1703 ms (client timeout 1000 ms). Trace:
I20260927 23:28:16.807617 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_39a61687324b4fcbbd0ab963fed8c694"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:16.810051 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_39a61687324b4fcbbd0ab963fed8c694 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:16.815169 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_d656464366cb481aa6c28f4294bd5a99"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:16.816813 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d656464366cb481aa6c28f4294bd5a99 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:16.818627 3167 tablet_bootstrap.cc:654] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:16.819025 2873 heartbeater.cc:347] Connected to a master server at 127.2.57.60:41571
I20260927 23:28:16.800222 3558 ts_tablet_manager.cc:1918] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:16.820091 3558 log.cc:1196] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/1da38499af164d7fb5c1a8d65167ad24
I20260927 23:28:16.820242 3963 raft_consensus.cc:482] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:16.820768 3963 raft_consensus.cc:504] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.822285 3558 ts_tablet_manager.cc:1939] T 1da38499af164d7fb5c1a8d65167ad24 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:16.823645 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb27269a590476f878ee52a2aefd141" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:16.824245 2824 raft_consensus.cc:2454] T 9eb27269a590476f878ee52a2aefd141 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:16.825281 2391 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_d31a1e740a2b42bd8569372845897579"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:16.825727 2609 leader_election.cc:302] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:16.826510 3965 raft_consensus.cc:2781] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:16.826838 3965 raft_consensus.cc:482] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:16.826957 3963 leader_election.cc:288] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:16.827140 3965 raft_consensus.cc:3037] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:16.827307 2391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d31a1e740a2b42bd8569372845897579 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:16.831735 2390 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_7340a0b8be8946d49f65fb36a2f6f09f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:16.833668 2390 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_7340a0b8be8946d49f65fb36a2f6f09f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:16.835569 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 1da38499af164d7fb5c1a8d65167ad24 (table default.Table_f7b5c90ef9a94341a88b0a40ea0f7e1e [id=b12aae65f933476ab57974d4b328a266]) successfully deleted
I20260927 23:28:16.835866 3965 raft_consensus.cc:504] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.836663 3558 tablet_replica.cc:330] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:16.836746 2388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_fe4babd936e548c7906748e495d790a1" } modify_external_catalogs: true
I20260927 23:28:16.837365 2388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_fe4babd936e548c7906748e495d790a1" } modify_external_catalogs: true
I20260927 23:28:16.844453 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb27269a590476f878ee52a2aefd141" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:16.845065 2824 raft_consensus.cc:3037] T 9eb27269a590476f878ee52a2aefd141 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:16.845714 3965 leader_election.cc:288] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:16.784633 2392 rpcz_store.cc:269] 0927 23:28:14.869552 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:15.501649 (+632097us) service_pool.cc:227] Handling call
0927 23:28:16.771145 (+1269496us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:16.185899 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:16.186289 (+ 390us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:16.186314 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:28:16.186327 (+ 13us) tablet.cc:660] Decoding operations
0927 23:28:16.186605 (+ 278us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:16.186647 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:16.186670 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:16.186829 (+ 159us) tablet.cc:699] Row locks acquired
0927 23:28:16.186842 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:28:16.186963 (+ 121us) write_op.cc:270] Start()
0927 23:28:16.187065 (+ 102us) write_op.cc:276] Timestamp: P: 1790551696186911 usec, L: 0
0927 23:28:16.187081 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:16.187589 (+ 508us) log.cc:842] Serialized 524 byte log entry
0927 23:28:16.197153 (+ 9564us) op_driver.cc:444] REPLICATION: finished
0927 23:28:16.197513 (+ 360us) write_op.cc:301] APPLY: starting
0927 23:28:16.197571 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:16.197685 (+ 114us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:16.197783 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:16.198018 (+ 235us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:16.198210 (+ 192us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:16.198257 (+ 47us) write_op.cc:312] APPLY: finished
0927 23:28:16.198491 (+ 234us) log.cc:842] Serialized 23 byte log entry
0927 23:28:16.198648 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:16.198720 (+ 72us) write_op.cc:454] Released schema lock
0927 23:28:16.198753 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:16.198942 (+ 189us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:16.198980 (+ 38us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":24,"child_traces":[["op",{"apply.queue_time_us":209,"apply.run_cpu_time_us":1628,"apply.run_wall_time_us":1627,"num_ops":1,"prepare.queue_time_us":110,"prepare.run_cpu_time_us":1977,"prepare.run_wall_time_us":1977,"raft.queue_time_us":1292,"raft.run_cpu_time_us":732,"raft.run_wall_time_us":731,"replication_time_us":9986}]]}
I20260927 23:28:16.852270 2824 raft_consensus.cc:2454] T 9eb27269a590476f878ee52a2aefd141 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
I20260927 23:28:16.853343 4005 raft_consensus.cc:482] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:16.853504 2609 leader_election.cc:302] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:16.854251 3965 raft_consensus.cc:2781] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:16.853844 4005 raft_consensus.cc:504] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.854683 3965 raft_consensus.cc:686] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Running, Role: LEADER
I20260927 23:28:16.855470 3965 consensus_queue.cc:237] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.845012 3558 raft_consensus.cc:2229] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:16.858263 3558 raft_consensus.cc:2258] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:16.861197 3558 ts_tablet_manager.cc:1905] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:16.866662 3167 tablet_bootstrap.cc:492] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:16.866571 4003 ts_tablet_manager.cc:1918] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:16.867002 4003 log.cc:1196] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/2411d7c53a154fafad9edda9fa75bb36
I20260927 23:28:16.867254 3167 ts_tablet_manager.cc:1397] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.092s user 0.007s sys 0.004s
I20260927 23:28:16.868672 4005 leader_election.cc:288] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:16.871585 2392 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_485140265fb4433280c5e7ae5316f122" } new_table_name: "default.Table_8eff0fa7f4994841bef399e5f67f7dca" modify_external_catalogs: true
I20260927 23:28:16.870666 3167 raft_consensus.cc:348] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.872095 3167 raft_consensus.cc:374] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:16.872361 3167 raft_consensus.cc:729] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:16.872936 3167 consensus_queue.cc:260] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.884534 3167 ts_tablet_manager.cc:1428] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.017s user 0.003s sys 0.003s
I20260927 23:28:16.867326 2669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0d8a1f277242588f57e4df95df5d5a" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:16.887459 2669 raft_consensus.cc:2454] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:28:16.888005 4003 ts_tablet_manager.cc:1939] T 2411d7c53a154fafad9edda9fa75bb36 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:16.889351 4003 tablet_replica.cc:330] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:16.889093 2763 leader_election.cc:302] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:16.889930 4004 raft_consensus.cc:2781] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:16.890250 4004 raft_consensus.cc:482] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:16.890322 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 2411d7c53a154fafad9edda9fa75bb36 (table default.Table_b967cf863a6848919c6542cb6bbb1c70 [id=597f1d374b754602850d1dbc3bd4e913]) successfully deleted
I20260927 23:28:16.890537 4004 raft_consensus.cc:3037] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:16.891261 2377 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9c86c7d874a49178c7b2cd7316afa14 (table default.Table_f45431c208e642ef9f406af42ff7d455 [id=d44851dff14d4e5084ae024995202140], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9c86c7d874a49178c7b2cd7316afa14 has no consensus state
I20260927 23:28:16.894433 4003 raft_consensus.cc:2229] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:16.897119 4003 raft_consensus.cc:2258] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:16.898238 4004 raft_consensus.cc:504] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.900036 4003 ts_tablet_manager.cc:1905] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:16.900574 3167 tablet_bootstrap.cc:492] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:16.904392 4004 leader_election.cc:288] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:16.906014 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0d8a1f277242588f57e4df95df5d5a" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:16.915234 2674 raft_consensus.cc:3037] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:16.923270 2674 raft_consensus.cc:2454] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:28:16.924743 2763 leader_election.cc:302] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:16.925997 4005 raft_consensus.cc:2781] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:16.926538 4005 raft_consensus.cc:686] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Becoming Leader. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Running, Role: LEADER
I20260927 23:28:16.927564 4005 consensus_queue.cc:237] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.944247 4003 ts_tablet_manager.cc:1918] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:16.952800 4003 log.cc:1196] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/1da38499af164d7fb5c1a8d65167ad24
I20260927 23:28:16.954340 4003 ts_tablet_manager.cc:1939] T 1da38499af164d7fb5c1a8d65167ad24 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:16.955089 3167 tablet_bootstrap.cc:654] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:16.797034 2387 rpcz_store.cc:269] 0927 23:28:15.093104 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:15.516644 (+423540us) service_pool.cc:227] Handling call
0927 23:28:15.539098 (+ 22454us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:16.796613 (+1257515us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:15.916453 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:15.916827 (+ 374us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:15.916854 (+ 27us) write_op.cc:435] Acquired schema lock
0927 23:28:15.916867 (+ 13us) tablet.cc:660] Decoding operations
0927 23:28:15.917298 (+ 431us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:15.917344 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:15.917365 (+ 21us) tablet.cc:683] Acquiring locks for 3 operations
0927 23:28:15.917606 (+ 241us) tablet.cc:699] Row locks acquired
0927 23:28:15.917620 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:15.917737 (+ 117us) write_op.cc:270] Start()
0927 23:28:15.917827 (+ 90us) write_op.cc:276] Timestamp: P: 1790551695917634 usec, L: 0
0927 23:28:15.917842 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:15.918358 (+ 516us) log.cc:842] Serialized 1266 byte log entry
0927 23:28:15.963227 (+ 44869us) op_driver.cc:444] REPLICATION: finished
0927 23:28:15.963497 (+ 270us) write_op.cc:301] APPLY: starting
0927 23:28:15.963557 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:15.963683 (+ 126us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:15.963803 (+ 120us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:15.964176 (+ 373us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:15.964369 (+ 193us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0927 23:28:15.964413 (+ 44us) write_op.cc:312] APPLY: finished
0927 23:28:15.964638 (+ 225us) log.cc:842] Serialized 35 byte log entry
0927 23:28:15.964804 (+ 166us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:15.964891 (+ 87us) write_op.cc:454] Released schema lock
0927 23:28:15.964924 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:16.020722 (+ 55798us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:16.020750 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":19,"spinlock_wait_cycles":35944704,"child_traces":[["op",{"apply.queue_time_us":77,"apply.run_cpu_time_us":1986,"apply.run_wall_time_us":57493,"mutex_wait_us":55521,"num_ops":3,"prepare.queue_time_us":124,"prepare.run_cpu_time_us":2186,"prepare.run_wall_time_us":3341,"raft.queue_time_us":1443,"raft.run_cpu_time_us":643,"raft.run_wall_time_us":643,"replication_time_us":45290}]]}
W20260927 23:28:16.957641 2763 consensus_peers.cc:487] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2411d7c53a154fafad9edda9fa75bb36. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:16.959719 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 1da38499af164d7fb5c1a8d65167ad24 (table default.Table_f7b5c90ef9a94341a88b0a40ea0f7e1e [id=b12aae65f933476ab57974d4b328a266]) successfully deleted
W20260927 23:28:16.961467 2380 catalog_manager.cc:4808] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 because the tablet was not found. No further retry: Not found: Tablet not found: 5bf856a05dff4fb4943fe8e1d9b7f385
I20260927 23:28:16.971437 3558 ts_tablet_manager.cc:1918] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:16.973474 3558 log.cc:1196] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/5bf856a05dff4fb4943fe8e1d9b7f385
I20260927 23:28:16.974123 3558 ts_tablet_manager.cc:1939] T 5bf856a05dff4fb4943fe8e1d9b7f385 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:16.983917 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 5bf856a05dff4fb4943fe8e1d9b7f385 (table default.Table_25f0e3650873407a8b3cbd8c8e0c86af [id=48f252a9cdb34eb1b452540f4186285c]) successfully deleted
I20260927 23:28:16.986752 3167 tablet_bootstrap.cc:492] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:16.987304 3167 ts_tablet_manager.cc:1397] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.087s user 0.012s sys 0.001s
I20260927 23:28:16.779557 2943 tablet_service.cc:1514] Processing DeleteTablet for tablet cfbbe17f54d1418b95efc3943db0b6c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:16.780097 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:16.780397 2934 tablet_service.cc:1514] Processing DeleteTablet for tablet fd0115095a82489a8aeb941efcc8a979 with delete_type TABLET_DATA_DELETED (replaced by tablet 178503861b1648d9a931d1606d807ed8) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:16.784492 2938 tablet_service.cc:1514] Processing DeleteTablet for tablet bbeeccc19a504e7984b293127766daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet e268bcb4b0614f37a4d9d96f8f40d1cb) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:16.991812 3558 tablet_replica.cc:330] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:16.992705 3558 raft_consensus.cc:2229] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:16.993770 3558 raft_consensus.cc:2258] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:16.995776 3558 ts_tablet_manager.cc:1905] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:16.795807 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da38499af164d7fb5c1a8d65167ad24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:14 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:16.870780 3072 tablet_bootstrap.cc:700] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent opening tablet: real 0.179s user 0.002s sys 0.000s
I20260927 23:28:16.998390 3072 tablet_bootstrap.cc:654] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:16.998659 3167 raft_consensus.cc:348] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:16.999481 3167 raft_consensus.cc:374] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:16.999773 3167 raft_consensus.cc:729] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:17.000655 3167 consensus_queue.cc:260] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:16.907965 2907 consensus_peers.cc:487] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:17.001686 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 93833.2us
I20260927 23:28:16.879142 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0d8a1f277242588f57e4df95df5d5a" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:17.003010 3167 ts_tablet_manager.cc:1428] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.015s user 0.003s sys 0.003s
I20260927 23:28:17.003908 3167 tablet_bootstrap.cc:492] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
W20260927 23:28:17.003985 2761 leader_election.cc:341] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:16.908522 2966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0d8a1f277242588f57e4df95df5d5a" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
W20260927 23:28:17.007408 2761 leader_election.cc:341] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:16.909117 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 196167us
I20260927 23:28:17.009243 3167 tablet_bootstrap.cc:654] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:17.009254 2608 leader_election.cc:341] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:17.009675 2608 leader_election.cc:302] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:17.011186 3963 raft_consensus.cc:2726] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:17.024864 3167 tablet_bootstrap.cc:492] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:17.025381 3167 ts_tablet_manager.cc:1397] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.007s
I20260927 23:28:17.027827 3167 raft_consensus.cc:348] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.028538 3167 raft_consensus.cc:374] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:17.028779 3167 raft_consensus.cc:729] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:16.711956 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 222) took 2623 ms. Trace:
I20260927 23:28:17.030786 3336 rpcz_store.cc:276] 0927 23:28:14.088424 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:14.088577 (+ 153us) service_pool.cc:227] Handling call
0927 23:28:16.711695 (+2623118us) ts_tablet_manager.cc:1145] Deleting tablet 2411d7c53a154fafad9edda9fa75bb36
0927 23:28:16.711736 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:16.711906 (+ 170us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2622418}
I20260927 23:28:17.031436 3167 consensus_queue.cc:260] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:17.032519 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 2411d7c53a154fafad9edda9fa75bb36 because the tablet was not found. No further retry: Not found: Tablet not found: 2411d7c53a154fafad9edda9fa75bb36
I20260927 23:28:17.035126 3167 ts_tablet_manager.cc:1428] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260927 23:28:17.041237 2966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb27269a590476f878ee52a2aefd141" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:17.041333 3998 tablet.cc:1722] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:17.042115 2966 raft_consensus.cc:2454] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 0.
I20260927 23:28:17.043124 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb27269a590476f878ee52a2aefd141" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:17.043763 2968 raft_consensus.cc:3037] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:17.046569 3167 tablet_bootstrap.cc:492] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:17.049957 2387 master_service.cc:946] ReplaceTablet: received request to replace tablet d9b629c1bbff4ea7b215b5b2c07d9f87 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:17.052930 3167 tablet_bootstrap.cc:654] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:17.058040 4016 tablet.cc:1722] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:17.062659 3558 ts_tablet_manager.cc:1918] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:17.063242 3558 log.cc:1196] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/2411d7c53a154fafad9edda9fa75bb36
I20260927 23:28:17.064680 3558 ts_tablet_manager.cc:1939] T 2411d7c53a154fafad9edda9fa75bb36 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:17.066635 3070 tablet_bootstrap.cc:654] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:17.068470 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 2411d7c53a154fafad9edda9fa75bb36 (table default.Table_b967cf863a6848919c6542cb6bbb1c70 [id=597f1d374b754602850d1dbc3bd4e913]) successfully deleted
W20260927 23:28:17.070863 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 702ebdeb7c5a4b7d900dbb00236cb8ae because the tablet was not found. No further retry: Not found: Tablet not found: 702ebdeb7c5a4b7d900dbb00236cb8ae
W20260927 23:28:17.073748 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet fd0115095a82489a8aeb941efcc8a979 because the tablet was not found. No further retry: Not found: Tablet not found: fd0115095a82489a8aeb941efcc8a979
I20260927 23:28:17.075030 4004 raft_consensus.cc:482] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:17.075510 4004 raft_consensus.cc:504] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.077580 4004 leader_election.cc:288] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:17.079265 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbeeccc19a504e7984b293127766daf0" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:17.079969 2674 raft_consensus.cc:2454] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:28:17.081301 2763 leader_election.cc:302] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:17.082190 4004 raft_consensus.cc:2781] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:17.082562 4004 raft_consensus.cc:482] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:17.082909 4004 raft_consensus.cc:3037] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:17.091542 3070 tablet_bootstrap.cc:492] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:17.092044 3070 ts_tablet_manager.cc:1397] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.495s user 0.012s sys 0.001s
I20260927 23:28:17.095969 3167 tablet_bootstrap.cc:492] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:17.096472 3167 ts_tablet_manager.cc:1397] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.061s user 0.018s sys 0.018s
I20260927 23:28:17.097504 4004 raft_consensus.cc:504] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.099515 3070 raft_consensus.cc:348] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.100189 3070 raft_consensus.cc:374] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:17.101250 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 223) took 2942 ms. Trace:
I20260927 23:28:17.101651 3336 rpcz_store.cc:276] 0927 23:28:14.158812 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:14.158951 (+ 139us) service_pool.cc:227] Handling call
0927 23:28:17.100801 (+2941850us) ts_tablet_manager.cc:1145] Deleting tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc
0927 23:28:17.100861 (+ 60us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:17.101184 (+ 323us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2867987}
W20260927 23:28:17.103039 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4ba28f8fbb6f4faf94f8820c9ba42fbc already in progress: creating tablet
I20260927 23:28:17.105305 4004 leader_election.cc:288] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:17.104245 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbeeccc19a504e7984b293127766daf0" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:17.106473 2674 raft_consensus.cc:3037] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:17.107424 2761 leader_election.cc:341] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: bbeeccc19a504e7984b293127766daf0
I20260927 23:28:17.098732 3167 raft_consensus.cc:348] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.112144 3167 raft_consensus.cc:374] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:17.113729 2674 raft_consensus.cc:2454] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:28:17.112475 3167 raft_consensus.cc:729] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
W20260927 23:28:17.114907 2761 leader_election.cc:341] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Not found: Tablet not found: bbeeccc19a504e7984b293127766daf0
I20260927 23:28:17.114758 3167 consensus_queue.cc:260] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.114944 2763 leader_election.cc:302] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:17.115906 3070 raft_consensus.cc:729] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:17.117774 3070 consensus_queue.cc:260] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.120100 4004 raft_consensus.cc:2781] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:17.120709 4004 raft_consensus.cc:686] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Becoming Leader. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Running, Role: LEADER
I20260927 23:28:17.121501 4004 consensus_queue.cc:237] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.123662 3070 ts_tablet_manager.cc:1428] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.031s user 0.006s sys 0.000s
I20260927 23:28:17.126142 3167 ts_tablet_manager.cc:1428] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.029s user 0.005s sys 0.000s
I20260927 23:28:17.127238 3167 tablet_bootstrap.cc:492] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:17.134997 3965 raft_consensus.cc:482] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:17.135490 3965 raft_consensus.cc:504] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.140130 3965 leader_election.cc:288] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:17.141409 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cd0972f635d4979abf5fd30e9b02b1d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:17.142740 2609 leader_election.cc:341] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:17.142997 3070 tablet_bootstrap.cc:492] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:17.144865 3167 tablet_bootstrap.cc:654] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:17.164949 3070 tablet_bootstrap.cc:654] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:17.178551 3072 tablet_bootstrap.cc:492] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:17.179018 3072 ts_tablet_manager.cc:1397] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.587s user 0.011s sys 0.000s
I20260927 23:28:17.183874 3167 tablet_bootstrap.cc:492] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:17.184362 3167 ts_tablet_manager.cc:1397] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.057s user 0.015s sys 0.000s
I20260927 23:28:17.186980 3167 raft_consensus.cc:348] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.187670 3167 raft_consensus.cc:374] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:17.187929 3167 raft_consensus.cc:729] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:17.193390 3167 consensus_queue.cc:260] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.196436 3167 ts_tablet_manager.cc:1428] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.012s user 0.004s sys 0.003s
I20260927 23:28:17.205875 3167 tablet_bootstrap.cc:492] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:17.214015 3167 tablet_bootstrap.cc:654] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:17.221109 3070 tablet_bootstrap.cc:492] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:17.221804 3070 ts_tablet_manager.cc:1397] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.088s user 0.004s sys 0.012s
I20260927 23:28:17.230000 3070 raft_consensus.cc:348] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.233330 3070 raft_consensus.cc:374] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:17.234372 3070 raft_consensus.cc:729] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:17.236760 3070 consensus_queue.cc:260] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.260262 3070 ts_tablet_manager.cc:1428] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.038s user 0.003s sys 0.002s
I20260927 23:28:17.266086 3070 tablet_bootstrap.cc:492] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:17.275477 2966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cd0972f635d4979abf5fd30e9b02b1d" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:17.307144 3070 tablet_bootstrap.cc:654] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:17.307600 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 224) took 3147 ms. Trace:
I20260927 23:28:17.307964 3336 rpcz_store.cc:276] 0927 23:28:14.160056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:14.160168 (+ 112us) service_pool.cc:227] Handling call
0927 23:28:17.179715 (+3019547us) ts_tablet_manager.cc:1145] Deleting tablet 0e0c12031a734ca0a6536404409c62ec
0927 23:28:17.179763 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:17.307531 (+127768us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":2946747}
W20260927 23:28:17.310166 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 0e0c12031a734ca0a6536404409c62ec because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0e0c12031a734ca0a6536404409c62ec already in progress: creating tablet
W20260927 23:28:17.311008 2906 consensus_peers.cc:487] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1da38499af164d7fb5c1a8d65167ad24. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:17.318203 3991 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.AlterSchema from 127.0.0.1:44952 (request call id 245) took 1174 ms. Trace:
I20260927 23:28:17.320504 3991 rpcz_store.cc:276] 0927 23:28:16.143706 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:16.153168 (+ 9462us) service_pool.cc:227] Handling call
0927 23:28:17.318128 (+1164960us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:16.201113 (+ 0us) alter_schema_op.cc:102] PREPARE ALTER-SCHEMA: Starting
0927 23:28:16.201467 (+ 354us) alter_schema_op.cc:63] Acquiring schema lock in exclusive mode
0927 23:28:16.201497 (+ 30us) alter_schema_op.cc:65] Acquired schema lock
0927 23:28:16.201546 (+ 49us) alter_schema_op.cc:117] PREPARE ALTER-SCHEMA: finished
0927 23:28:16.201663 (+ 117us) alter_schema_op.cc:126] START. Timestamp: 1790551696201599
0927 23:28:16.201682 (+ 19us) op_driver.cc:331] REPLICATION: starting
0927 23:28:16.202223 (+ 541us) log.cc:842] Serialized 173 byte log entry
0927 23:28:16.380483 (+178260us) op_driver.cc:444] REPLICATION: finished
0927 23:28:16.394420 (+ 13937us) alter_schema_op.cc:131] APPLY ALTER-SCHEMA: Starting
0927 23:28:16.582054 (+187634us) log.cc:842] Serialized 5 byte log entry
0927 23:28:17.306070 (+724016us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:17.317879 (+ 11809us) log.cc:842] Serialized 14 byte log entry
0927 23:28:17.317974 (+ 95us) alter_schema_op.cc:71] Released schema lock
0927 23:28:17.317998 (+ 24us) alter_schema_op.cc:184] AlterSchemaCommitCallback: making alter schema visible
Metrics: {"child_traces":[["op",{"apply.queue_time_us":13690,"prepare.queue_time_us":154,"prepare.run_cpu_time_us":7968,"prepare.run_wall_time_us":49343,"raft.queue_time_us":2522,"raft.run_cpu_time_us":1736,"raft.run_wall_time_us":2999,"replication_time_us":178723,"thread_start_us":85589,"threads_started":2,"wal-append.queue_time_us":42646}]]}
I20260927 23:28:17.346771 4004 raft_consensus.cc:482] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:17.336424 3072 raft_consensus.cc:348] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.347620 3072 raft_consensus.cc:374] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:17.347329 4004 raft_consensus.cc:504] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.347965 3072 raft_consensus.cc:729] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:17.337500 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 230) took 1852 ms. Trace:
I20260927 23:28:17.349468 4004 leader_election.cc:288] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:17.349601 3336 rpcz_store.cc:276] 0927 23:28:15.484721 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:15.484886 (+ 165us) service_pool.cc:227] Handling call
0927 23:28:17.309231 (+1824345us) ts_tablet_manager.cc:1145] Deleting tablet 78e099028f7847d1b3de7d437be97de0
0927 23:28:17.309277 (+ 46us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:17.337393 (+ 28116us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1823710}
I20260927 23:28:17.348855 3072 consensus_queue.cc:260] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.351277 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c87d497da884544920b3be9c78daccf" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:17.351577 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 78e099028f7847d1b3de7d437be97de0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 78e099028f7847d1b3de7d437be97de0 already in progress: creating tablet
I20260927 23:28:17.352455 2966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c87d497da884544920b3be9c78daccf" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:17.353286 3072 ts_tablet_manager.cc:1428] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.174s user 0.007s sys 0.000s
I20260927 23:28:17.353879 2761 leader_election.cc:302] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:17.354239 3072 tablet_bootstrap.cc:492] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:17.354923 4004 raft_consensus.cc:2726] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:17.355477 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 231) took 1842 ms. Trace:
I20260927 23:28:17.355880 3336 rpcz_store.cc:276] 0927 23:28:15.512862 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:15.513065 (+ 203us) service_pool.cc:227] Handling call
0927 23:28:17.350746 (+1837681us) ts_tablet_manager.cc:1145] Deleting tablet e81df7c7f1a04eb9904dd2a1a5a596c4
0927 23:28:17.350797 (+ 51us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:17.355419 (+ 4622us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1837038}
W20260927 23:28:17.357335 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet e81df7c7f1a04eb9904dd2a1a5a596c4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e81df7c7f1a04eb9904dd2a1a5a596c4 already in progress: creating tablet
I20260927 23:28:17.341566 2968 raft_consensus.cc:2454] T 9eb27269a590476f878ee52a2aefd141 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in term 1.
W20260927 23:28:17.363035 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 46072.8us
I20260927 23:28:17.362875 4001 consensus_queue.cc:1035] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:28:17.364075 2608 leader_election.cc:341] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:17.364636 2608 leader_election.cc:302] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:17.365587 4001 raft_consensus.cc:2726] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:17.386842 2391 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 295) took 2613 ms. Trace:
I20260927 23:28:17.387217 2391 rpcz_store.cc:276] 0927 23:28:14.773419 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:16.824756 (+2051337us) service_pool.cc:227] Handling call
0927 23:28:16.827679 (+ 2923us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:16.829133 (+ 1454us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:17.386141 (+557008us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:17.386481 (+ 340us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:17.386775 (+ 294us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:17.094900 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:17.095362 (+ 462us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:17.095389 (+ 27us) write_op.cc:435] Acquired schema lock
0927 23:28:17.095404 (+ 15us) tablet.cc:660] Decoding operations
0927 23:28:17.095723 (+ 319us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:17.095785 (+ 62us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:17.095810 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:17.096048 (+ 238us) tablet.cc:699] Row locks acquired
0927 23:28:17.096097 (+ 49us) write_op.cc:260] PREPARE: finished
0927 23:28:17.096245 (+ 148us) write_op.cc:270] Start()
0927 23:28:17.096361 (+ 116us) write_op.cc:276] Timestamp: P: 1790551697096157 usec, L: 0
0927 23:28:17.096380 (+ 19us) op_driver.cc:331] REPLICATION: starting
0927 23:28:17.096996 (+ 616us) log.cc:842] Serialized 433 byte log entry
0927 23:28:17.250082 (+153086us) op_driver.cc:444] REPLICATION: finished
0927 23:28:17.379819 (+129737us) write_op.cc:301] APPLY: starting
0927 23:28:17.379886 (+ 67us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:17.380004 (+ 118us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:17.384418 (+ 4414us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:17.384974 (+ 556us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:17.385132 (+ 158us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:17.385175 (+ 43us) write_op.cc:312] APPLY: finished
0927 23:28:17.385405 (+ 230us) log.cc:842] Serialized 29 byte log entry
0927 23:28:17.385573 (+ 168us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.385639 (+ 66us) write_op.cc:454] Released schema lock
0927 23:28:17.385689 (+ 50us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:17.385899 (+ 210us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.385922 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":129513,"apply.run_cpu_time_us":2046,"apply.run_wall_time_us":6300,"num_ops":2,"prepare.queue_time_us":264320,"prepare.run_cpu_time_us":2434,"prepare.run_wall_time_us":3435,"raft.queue_time_us":82,"raft.run_cpu_time_us":714,"raft.run_wall_time_us":713,"replication_time_us":153612,"spinlock_wait_cycles":211968,"thread_start_us":327748,"threads_started":2}]]}
I20260927 23:28:17.432382 3938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:28:17.433072 3938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:28:17.437688 3167 tablet_bootstrap.cc:492] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:17.438951 3167 ts_tablet_manager.cc:1397] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.233s user 0.073s sys 0.125s
I20260927 23:28:17.441912 3167 raft_consensus.cc:348] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.443348 2393 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 294) took 2690 ms. Trace:
I20260927 23:28:17.443703 2393 rpcz_store.cc:276] 0927 23:28:14.753049 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:16.814691 (+2061642us) service_pool.cc:227] Handling call
0927 23:28:16.817229 (+ 2538us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:16.818720 (+ 1491us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:17.442636 (+623916us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:17.442977 (+ 341us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:17.443273 (+ 296us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:17.098603 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:17.099020 (+ 417us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:17.099048 (+ 28us) write_op.cc:435] Acquired schema lock
0927 23:28:17.099063 (+ 15us) tablet.cc:660] Decoding operations
0927 23:28:17.099377 (+ 314us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:17.099416 (+ 39us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:17.099443 (+ 27us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:17.099653 (+ 210us) tablet.cc:699] Row locks acquired
0927 23:28:17.099669 (+ 16us) write_op.cc:260] PREPARE: finished
0927 23:28:17.099786 (+ 117us) write_op.cc:270] Start()
0927 23:28:17.099874 (+ 88us) write_op.cc:276] Timestamp: P: 1790551697099701 usec, L: 0
0927 23:28:17.099893 (+ 19us) op_driver.cc:331] REPLICATION: starting
0927 23:28:17.248517 (+148624us) log.cc:842] Serialized 433 byte log entry
0927 23:28:17.371966 (+123449us) op_driver.cc:444] REPLICATION: finished
0927 23:28:17.386216 (+ 14250us) write_op.cc:301] APPLY: starting
0927 23:28:17.386270 (+ 54us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:17.386369 (+ 99us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:17.386464 (+ 95us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:17.386854 (+ 390us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:17.386978 (+ 124us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:17.387019 (+ 41us) write_op.cc:312] APPLY: finished
0927 23:28:17.387193 (+ 174us) log.cc:842] Serialized 29 byte log entry
0927 23:28:17.387326 (+ 133us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.387384 (+ 58us) write_op.cc:454] Released schema lock
0927 23:28:17.387412 (+ 28us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":14108,"num_ops":2,"prepare.queue_time_us":188054,"prepare.run_cpu_time_us":2303,"prepare.run_wall_time_us":153208,"raft.queue_time_us":80,"raft.run_cpu_time_us":709,"raft.run_wall_time_us":710,"replication_time_us":271985,"threads_started":1}]]}
I20260927 23:28:17.447825 2393 master_service.cc:946] ReplaceTablet: received request to replace tablet c848ed8a221a4b799529ccc0d1830893 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:17.440001 3902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19a88ff0709d4266b933cc526577127f. Latest consensus state: current_term: 3 leader_uuid: "19a88ff0709d4266b933cc526577127f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:28:17.453792 3902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:28:17.450675 2824 raft_consensus.cc:3037] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:17.464289 3821 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:46208 (request call id 0) took 3436 ms. Trace:
I20260927 23:28:17.465262 3167 raft_consensus.cc:374] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:17.465566 3167 raft_consensus.cc:729] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:17.466058 3621 consensus_queue.cc:1035] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:17.466189 3167 consensus_queue.cc:260] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.471304 3938 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987)
I20260927 23:28:17.478762 3167 ts_tablet_manager.cc:1428] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.039s user 0.006s sys 0.000s
I20260927 23:28:17.479802 3167 tablet_bootstrap.cc:492] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:17.491451 3070 tablet_bootstrap.cc:492] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:17.492084 3070 ts_tablet_manager.cc:1397] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.226s user 0.016s sys 0.022s
I20260927 23:28:17.492246 3167 tablet_bootstrap.cc:654] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:17.493273 3991 tablet.cc:1722] T 4eff8f3e5ef74e93b077b5e5bfb139c6 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:17.496845 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 233) took 1858 ms. Trace:
I20260927 23:28:17.497205 3336 rpcz_store.cc:276] 0927 23:28:15.638741 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:15.638894 (+ 153us) service_pool.cc:227] Handling call
0927 23:28:17.496445 (+1857551us) ts_tablet_manager.cc:1145] Deleting tablet 3a8dbc1165ac4f5b8ed792f6064cce6f
0927 23:28:17.496494 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:17.496784 (+ 290us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1807938}
W20260927 23:28:17.498543 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 3a8dbc1165ac4f5b8ed792f6064cce6f because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a8dbc1165ac4f5b8ed792f6064cce6f already in progress: creating tablet
I20260927 23:28:17.508064 2393 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:51972 (request call id 300) took 1241 ms. Trace:
I20260927 23:28:17.508559 2393 rpcz_store.cc:276] 0927 23:28:16.266743 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:17.447689 (+1180946us) service_pool.cc:227] Handling call
0927 23:28:17.507314 (+ 59625us) spinlock_profiling.cc:243] Waited 58.5 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b5b81 00007f8bb3c10d37 00007f8bb3c10cc2 00007f8bb3c10bf1 00007f8bb3c10b6c 00007f8bb3c0fe22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:17.507985 (+ 671us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":134645632}
I20260927 23:28:17.511111 2393 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_6568f80896124508811607305bf18ff9" } new_table_name: "default.Table_4458b554cada4d42bc1307905a783181" modify_external_catalogs: true
W20260927 23:28:17.528420 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet cfbbe17f54d1418b95efc3943db0b6c1 because the tablet was not found. No further retry: Not found: Tablet not found: cfbbe17f54d1418b95efc3943db0b6c1
W20260927 23:28:17.530912 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 because the tablet was not found. No further retry: Not found: Tablet not found: 5bf856a05dff4fb4943fe8e1d9b7f385
W20260927 23:28:17.532352 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet fd0115095a82489a8aeb941efcc8a979 because the tablet was not found. No further retry: Not found: Tablet not found: fd0115095a82489a8aeb941efcc8a979
I20260927 23:28:17.537184 3963 consensus_queue.cc:1035] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:17.556912 3572 consensus_queue.cc:1035] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260927 23:28:17.560122 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet bbeeccc19a504e7984b293127766daf0 because the tablet was not found. No further retry: Not found: Tablet not found: bbeeccc19a504e7984b293127766daf0
I20260927 23:28:17.561233 3336 tablet_replica.cc:330] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:17.562250 3336 raft_consensus.cc:2229] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:17.574213 4034 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 23:28:17.579612 3072 tablet_bootstrap.cc:654] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:17.594156 3336 raft_consensus.cc:2258] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:17.596691 3336 ts_tablet_manager.cc:1905] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:17.606333 3762 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" instance_seqno: 1790551674204875) as {username='slave'} at 127.2.57.3:39103; Asking this server to re-register.
W20260927 23:28:17.662985 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 89881.3us
I20260927 23:28:17.665695 3336 ts_tablet_manager.cc:1918] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:17.666100 3336 log.cc:1196] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/1da38499af164d7fb5c1a8d65167ad24
I20260927 23:28:17.666807 3336 ts_tablet_manager.cc:1939] T 1da38499af164d7fb5c1a8d65167ad24 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:17.668726 3017 heartbeater.cc:464] Registering TS with master...
I20260927 23:28:17.679589 2390 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 296) took 2757 ms. Trace:
I20260927 23:28:17.684413 2389 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:51972 (request call id 293) took 2934 ms. Trace:
I20260927 23:28:17.687521 2387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet d9b629c1bbff4ea7b215b5b2c07d9f87
I20260927 23:28:17.689126 2387 catalog_manager.cc:6565] ReplaceTablet: tablet d9b629c1bbff4ea7b215b5b2c07d9f87 deleted and replaced by tablet d6c89691b3734d94a358fb394994d3d3
I20260927 23:28:17.689467 2387 rpcz_store.cc:275] Call kudu.master.MasterService.ReplaceTablet from 127.0.0.1:51972 (request call id 299) took 1732 ms. Trace:
I20260927 23:28:17.690843 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet d9b629c1bbff4ea7b215b5b2c07d9f87 with delete_type TABLET_DATA_DELETED (replaced by tablet d6c89691b3734d94a358fb394994d3d3) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:17.679965 2390 rpcz_store.cc:276] 0927 23:28:14.922126 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:16.831070 (+1908944us) service_pool.cc:227] Handling call
0927 23:28:16.834127 (+ 3057us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:16.963402 (+129275us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:17.641957 (+678555us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:17.679188 (+ 37231us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:17.679514 (+ 326us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:17.519682 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:17.520108 (+ 426us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:17.520133 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:28:17.520149 (+ 16us) tablet.cc:660] Decoding operations
0927 23:28:17.520471 (+ 322us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:17.520508 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:17.520531 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:17.520741 (+ 210us) tablet.cc:699] Row locks acquired
0927 23:28:17.520923 (+ 182us) write_op.cc:260] PREPARE: finished
0927 23:28:17.521054 (+ 131us) write_op.cc:270] Start()
0927 23:28:17.521145 (+ 91us) write_op.cc:276] Timestamp: P: 1790551697520965 usec, L: 0
0927 23:28:17.521163 (+ 18us) op_driver.cc:331] REPLICATION: starting
0927 23:28:17.521646 (+ 483us) log.cc:842] Serialized 433 byte log entry
0927 23:28:17.620649 (+ 99003us) op_driver.cc:444] REPLICATION: finished
0927 23:28:17.637997 (+ 17348us) write_op.cc:301] APPLY: starting
0927 23:28:17.638250 (+ 253us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:17.638576 (+ 326us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:17.638680 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:17.639141 (+ 461us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:17.640199 (+ 1058us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:17.640255 (+ 56us) write_op.cc:312] APPLY: finished
0927 23:28:17.640860 (+ 605us) log.cc:842] Serialized 29 byte log entry
0927 23:28:17.641050 (+ 190us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.641126 (+ 76us) write_op.cc:454] Released schema lock
0927 23:28:17.641176 (+ 50us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:17.643106 (+ 1930us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.643203 (+ 97us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":17163,"apply.run_cpu_time_us":2357,"apply.run_wall_time_us":6117,"num_ops":2,"prepare.queue_time_us":554746,"prepare.run_cpu_time_us":2109,"prepare.run_wall_time_us":3473,"replication_time_us":99416}]]}
I20260927 23:28:17.694152 4004 raft_consensus.cc:482] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:17.694648 4004 raft_consensus.cc:504] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.689751 2387 rpcz_store.cc:276] 0927 23:28:15.957153 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:17.049753 (+1092600us) service_pool.cc:227] Handling call
0927 23:28:17.689411 (+639658us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:17.586528 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:17.586977 (+ 449us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:17.587003 (+ 26us) write_op.cc:435] Acquired schema lock
0927 23:28:17.587019 (+ 16us) tablet.cc:660] Decoding operations
0927 23:28:17.587363 (+ 344us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:17.587414 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:17.587446 (+ 32us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:17.587675 (+ 229us) tablet.cc:699] Row locks acquired
0927 23:28:17.587693 (+ 18us) write_op.cc:260] PREPARE: finished
0927 23:28:17.587834 (+ 141us) write_op.cc:270] Start()
0927 23:28:17.587921 (+ 87us) write_op.cc:276] Timestamp: P: 1790551697587740 usec, L: 0
0927 23:28:17.587940 (+ 19us) op_driver.cc:331] REPLICATION: starting
0927 23:28:17.588573 (+ 633us) log.cc:842] Serialized 619 byte log entry
0927 23:28:17.684389 (+ 95816us) op_driver.cc:444] REPLICATION: finished
0927 23:28:17.685045 (+ 656us) write_op.cc:301] APPLY: starting
0927 23:28:17.685111 (+ 66us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:17.685316 (+ 205us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:17.685481 (+ 165us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:17.685920 (+ 439us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:17.686176 (+ 256us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:17.686224 (+ 48us) write_op.cc:312] APPLY: finished
0927 23:28:17.686450 (+ 226us) log.cc:842] Serialized 29 byte log entry
0927 23:28:17.686628 (+ 178us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.686701 (+ 73us) write_op.cc:454] Released schema lock
0927 23:28:17.686742 (+ 41us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":466,"num_ops":2,"prepare.queue_time_us":385443,"prepare.run_cpu_time_us":2341,"prepare.run_wall_time_us":15961,"replication_time_us":96329}]]}
I20260927 23:28:17.697387 4004 leader_election.cc:288] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:17.699738 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet d9b629c1bbff4ea7b215b5b2c07d9f87 with delete_type TABLET_DATA_DELETED (replaced by tablet d6c89691b3734d94a358fb394994d3d3) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:17.704726 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4a9d69d75c4bfbb43e558b83b460d1" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:17.705508 2971 raft_consensus.cc:2379] T 9f4a9d69d75c4bfbb43e558b83b460d1 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate abe182b2afc64d908f692ef45eae1d51 in current term 1: Already voted for candidate 5ce671f028a34ea4b8f0520c2689c9a1 in this term.
W20260927 23:28:17.709129 2906 consensus_peers.cc:487] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:17.652762 3070 raft_consensus.cc:348] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.714943 3070 raft_consensus.cc:374] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:17.715255 3070 raft_consensus.cc:729] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:17.666365 4043 raft_consensus.cc:482] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:17.684955 2389 rpcz_store.cc:276] 0927 23:28:14.749236 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:16.807083 (+2057847us) service_pool.cc:227] Handling call
0927 23:28:16.810478 (+ 3395us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:16.812324 (+ 1846us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:17.617650 (+805326us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:17.683332 (+ 65682us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:17.683657 (+ 325us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:17.251943 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:17.252414 (+ 471us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:17.252439 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:28:17.252450 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:17.252784 (+ 334us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:17.252827 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:17.252850 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:17.253078 (+ 228us) tablet.cc:699] Row locks acquired
0927 23:28:17.253095 (+ 17us) write_op.cc:260] PREPARE: finished
0927 23:28:17.253224 (+ 129us) write_op.cc:270] Start()
0927 23:28:17.253320 (+ 96us) write_op.cc:276] Timestamp: P: 1790551697253165 usec, L: 0
0927 23:28:17.253337 (+ 17us) op_driver.cc:331] REPLICATION: starting
0927 23:28:17.512922 (+259585us) log.cc:842] Serialized 433 byte log entry
0927 23:28:17.514697 (+ 1775us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068471 00007f8bb40df198 00007f8bb4540ade 00007f8bb45402b8 00007f8bb4544ba1 00007f8bb4544b59 00007f8bb4544ae9 00007f8bb4544ab1 00007f8bb4543dad 0000000000580784 00000000005805b9
0927 23:28:17.602554 (+ 87857us) op_driver.cc:444] REPLICATION: finished
0927 23:28:17.616030 (+ 13476us) write_op.cc:301] APPLY: starting
0927 23:28:17.616097 (+ 67us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:17.616214 (+ 117us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:17.616234 (+ 20us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:17.616755 (+ 521us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:17.616900 (+ 145us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:17.616940 (+ 40us) write_op.cc:312] APPLY: finished
0927 23:28:17.617164 (+ 224us) log.cc:842] Serialized 29 byte log entry
0927 23:28:17.617338 (+ 174us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.617405 (+ 67us) write_op.cc:454] Released schema lock
0927 23:28:17.617434 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:17.618068 (+ 634us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.618115 (+ 47us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":13268,"apply.run_cpu_time_us":1906,"apply.run_wall_time_us":2373,"num_ops":2,"prepare.queue_time_us":339383,"prepare.run_cpu_time_us":4214,"prepare.run_wall_time_us":263824,"raft.queue_time_us":89,"raft.run_cpu_time_us":669,"raft.run_wall_time_us":669,"replication_time_us":349121,"spinlock_wait_cycles":595534720}]]}
I20260927 23:28:17.689790 2653 tablet_service.cc:1514] Processing DeleteTablet for tablet d9b629c1bbff4ea7b215b5b2c07d9f87 with delete_type TABLET_DATA_DELETED (replaced by tablet d6c89691b3734d94a358fb394994d3d3) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:17.719261 4043 raft_consensus.cc:504] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.720535 3762 ts_manager.cc:194] Registered new tserver with Master: de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:17.721436 4043 leader_election.cc:288] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:17.715909 3070 consensus_queue.cc:260] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.723414 3070 ts_tablet_manager.cc:1428] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.231s user 0.006s sys 0.000s
I20260927 23:28:17.724864 3070 tablet_bootstrap.cc:492] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:17.731218 2389 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_8823e2b699aa4e13badf61047486415f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:17.735414 3072 tablet_bootstrap.cc:492] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
W20260927 23:28:17.749182 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 183290us
I20260927 23:28:17.753680 3756 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" instance_seqno: 1790551672306438) as {username='slave'} at 127.2.57.2:34611; Asking this server to re-register.
I20260927 23:28:17.759514 3167 tablet_bootstrap.cc:492] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:17.760010 3167 ts_tablet_manager.cc:1397] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.280s user 0.091s sys 0.106s
I20260927 23:28:17.760100 3070 tablet_bootstrap.cc:654] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:17.761096 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119a1338b04740d0bbf291c8fbe34992" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:17.762607 3167 raft_consensus.cc:348] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.763221 3167 raft_consensus.cc:374] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:17.763484 2873 heartbeater.cc:464] Registering TS with master...
I20260927 23:28:17.763440 3167 raft_consensus.cc:729] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:17.764401 3167 consensus_queue.cc:260] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.734050 2387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_c632a6c33818436ca316f423eb2b3aa7" } modify_external_catalogs: true
I20260927 23:28:17.769037 2387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_c632a6c33818436ca316f423eb2b3aa7" } modify_external_catalogs: true
I20260927 23:28:17.773553 3167 ts_tablet_manager.cc:1428] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260927 23:28:17.783598 3167 tablet_bootstrap.cc:492] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
W20260927 23:28:17.792689 2389 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_8823e2b699aa4e13badf61047486415f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:17.793612 3072 ts_tablet_manager.cc:1397] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.440s user 0.015s sys 0.011s
I20260927 23:28:17.793777 4004 consensus_queue.cc:1035] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:17.766613 3336 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44952 (request call id 248) took 1013 ms. Trace:
I20260927 23:28:17.805426 3336 rpcz_store.cc:276] 0927 23:28:16.753495 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:16.795627 (+ 42132us) service_pool.cc:227] Handling call
0927 23:28:17.561068 (+765441us) ts_tablet_manager.cc:1145] Deleting tablet 1da38499af164d7fb5c1a8d65167ad24
0927 23:28:17.561111 (+ 43us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:17.643542 (+ 82431us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:17.665416 (+ 21874us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:17.766546 (+101130us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":563648}
I20260927 23:28:17.807013 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 1da38499af164d7fb5c1a8d65167ad24 (table default.Table_f7b5c90ef9a94341a88b0a40ea0f7e1e [id=b12aae65f933476ab57974d4b328a266]) successfully deleted
I20260927 23:28:17.811843 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119a1338b04740d0bbf291c8fbe34992" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:28:17.814877 2380 catalog_manager.cc:4821] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): delete failed for tablet d9b629c1bbff4ea7b215b5b2c07d9f87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9b629c1bbff4ea7b215b5b2c07d9f87 already in progress: creating tablet
W20260927 23:28:17.815826 2761 consensus_peers.cc:487] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 -> Peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Couldn't send request to peer de216a4a8e4b4e1b857d8647a3b05f6d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:17.791347 2609 leader_election.cc:341] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:17.792497 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f4a9d69d75c4bfbb43e558b83b460d1" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:17.829339 2763 leader_election.cc:302] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:17.830672 4005 raft_consensus.cc:3037] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:17.838519 4005 raft_consensus.cc:2726] T 9f4a9d69d75c4bfbb43e558b83b460d1 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:17.840153 3621 consensus_queue.cc:1035] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:17.795985 3072 raft_consensus.cc:348] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.842978 3072 raft_consensus.cc:374] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:17.843281 3072 raft_consensus.cc:729] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:17.843964 3072 consensus_queue.cc:260] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.846026 3072 ts_tablet_manager.cc:1428] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.052s user 0.005s sys 0.000s
W20260927 23:28:17.846743 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet d9b629c1bbff4ea7b215b5b2c07d9f87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9b629c1bbff4ea7b215b5b2c07d9f87 already in progress: creating tablet
I20260927 23:28:17.857388 3167 tablet_bootstrap.cc:654] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:17.859265 2384 catalog_manager.cc:5495] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: HEALTHY } } }
W20260927 23:28:17.861502 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet d9b629c1bbff4ea7b215b5b2c07d9f87 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet d9b629c1bbff4ea7b215b5b2c07d9f87 already in progress: creating tablet
I20260927 23:28:17.861696 2384 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af]). Expected version 2 got 0
I20260927 23:28:17.862295 2384 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:17.866341 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 850dde60e3b147c28a7668a177f75aec (DEFAULT_TABLE table=default.Table_a66ee72e2abf4d8ca5f23c5ff0306640 [id=b0892b94eea1478cb04d095c2f4ec616]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:17.866598 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 850dde60e3b147c28a7668a177f75aec (DEFAULT_TABLE table=default.Table_a66ee72e2abf4d8ca5f23c5ff0306640 [id=b0892b94eea1478cb04d095c2f4ec616]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:17.867434 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850dde60e3b147c28a7668a177f75aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:17.867605 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850dde60e3b147c28a7668a177f75aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:17.882592 2653 tablet_service.cc:1467] Processing CreateTablet for tablet 850dde60e3b147c28a7668a177f75aec (DEFAULT_TABLE table=default.Table_a66ee72e2abf4d8ca5f23c5ff0306640 [id=b0892b94eea1478cb04d095c2f4ec616]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:17.883759 2653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 850dde60e3b147c28a7668a177f75aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:17.892489 3072 tablet_bootstrap.cc:492] T 50dd9944d3134ab9b23a6109fd7d6ee5 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:17.902552 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 6bd8b531180f48d59cf8a90f3ea82149 (DEFAULT_TABLE table=default.Table_0fd5cd6b3a5c4d0598149d887222ad33 [id=8269b22e14ea4f148e62adfeb58c27ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:17.903678 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bd8b531180f48d59cf8a90f3ea82149. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:17.917110 2608 leader_election.cc:341] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:17.917590 2608 leader_election.cc:302] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:17.900063 2384 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 7f530db3b52944f6a283ee9e2bf66828 (table default.Table_55dca2c970f045b6966759363c4a0816 [id=ce182a32bf8a42e8990fbafc47dedc14]). Expected version 2 got 0
I20260927 23:28:17.921252 4005 raft_consensus.cc:482] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:17.921856 4005 raft_consensus.cc:504] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.923348 3072 tablet_bootstrap.cc:654] T 50dd9944d3134ab9b23a6109fd7d6ee5 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:17.925513 2966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49883297d2b046b99f2ec1343c276ff9" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:17.927084 2966 raft_consensus.cc:2454] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:28:17.930413 2938 tablet_service.cc:1467] Processing CreateTablet for tablet 6bd8b531180f48d59cf8a90f3ea82149 (DEFAULT_TABLE table=default.Table_0fd5cd6b3a5c4d0598149d887222ad33 [id=8269b22e14ea4f148e62adfeb58c27ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:17.931599 2938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bd8b531180f48d59cf8a90f3ea82149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:17.933033 2761 leader_election.cc:302] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:17.933910 4004 raft_consensus.cc:2781] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:17.934239 4004 raft_consensus.cc:482] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:17.934514 4004 raft_consensus.cc:3037] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:17.936213 3621 raft_consensus.cc:482] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:17.936699 3621 raft_consensus.cc:504] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.938663 3621 leader_election.cc:288] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:17.939790 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cd0972f635d4979abf5fd30e9b02b1d" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
W20260927 23:28:17.941248 2906 leader_election.cc:341] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:17.945541 2377 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9c86c7d874a49178c7b2cd7316afa14 (table default.Table_f45431c208e642ef9f406af42ff7d455 [id=d44851dff14d4e5084ae024995202140], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9c86c7d874a49178c7b2cd7316afa14 consensus state has no leader
I20260927 23:28:17.947081 4004 raft_consensus.cc:504] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.947710 2802 tablet_service.cc:1467] Processing CreateTablet for tablet 4fe36a27814640a1a783a324189d46e2 (DEFAULT_TABLE table=default.Table_b5b37ba466074efda0f8867fc0af9e50 [id=379b3d950a514d52807930d157d01b38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:17.948776 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fe36a27814640a1a783a324189d46e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:17.949501 4005 leader_election.cc:288] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:17.949437 2934 tablet_service.cc:1467] Processing CreateTablet for tablet 4fe36a27814640a1a783a324189d46e2 (DEFAULT_TABLE table=default.Table_b5b37ba466074efda0f8867fc0af9e50 [id=379b3d950a514d52807930d157d01b38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:17.950657 2934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fe36a27814640a1a783a324189d46e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:17.952562 2966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49883297d2b046b99f2ec1343c276ff9" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:17.953161 2966 raft_consensus.cc:3037] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:17.965911 2647 tablet_service.cc:1467] Processing CreateTablet for tablet 6bd8b531180f48d59cf8a90f3ea82149 (DEFAULT_TABLE table=default.Table_0fd5cd6b3a5c4d0598149d887222ad33 [id=8269b22e14ea4f148e62adfeb58c27ce]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:17.966073 4048 raft_consensus.cc:2726] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:17.968219 3167 tablet_bootstrap.cc:492] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:17.968725 3167 ts_tablet_manager.cc:1397] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.185s user 0.019s sys 0.003s
I20260927 23:28:17.968562 2966 raft_consensus.cc:2454] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:28:17.970577 2761 leader_election.cc:302] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:17.971290 3167 raft_consensus.cc:348] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:17.972026 3167 raft_consensus.cc:374] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:17.972270 3167 raft_consensus.cc:729] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:17.972810 4005 raft_consensus.cc:2781] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:17.922138 2384 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
I20260927 23:28:17.981171 2384 catalog_manager.cc:5495] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: HEALTHY } } }
I20260927 23:28:17.982820 2384 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:17.983515 2384 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 9eb27269a590476f878ee52a2aefd141 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]). Expected version 1 got 0
I20260927 23:28:17.984647 2668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49883297d2b046b99f2ec1343c276ff9" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:17.985246 2668 raft_consensus.cc:3037] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:17.988895 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49883297d2b046b99f2ec1343c276ff9" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:17.973085 3167 consensus_queue.cc:260] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.001592 3167 ts_tablet_manager.cc:1428] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.033s user 0.007s sys 0.000s
I20260927 23:28:18.002055 4004 leader_election.cc:288] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:18.004156 2947 tablet_service.cc:1467] Processing CreateTablet for tablet b9c86c7d874a49178c7b2cd7316afa14 (DEFAULT_TABLE table=default.Table_f45431c208e642ef9f406af42ff7d455 [id=d44851dff14d4e5084ae024995202140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.005342 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c86c7d874a49178c7b2cd7316afa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.026870 2649 tablet_service.cc:1467] Processing CreateTablet for tablet b9c86c7d874a49178c7b2cd7316afa14 (DEFAULT_TABLE table=default.Table_f45431c208e642ef9f406af42ff7d455 [id=d44851dff14d4e5084ae024995202140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.025671 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 4fe36a27814640a1a783a324189d46e2 (DEFAULT_TABLE table=default.Table_b5b37ba466074efda0f8867fc0af9e50 [id=379b3d950a514d52807930d157d01b38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.045917 2943 tablet_service.cc:1467] Processing CreateTablet for tablet aea80e2e708c41b78dc5433faa34b0f5 (DEFAULT_TABLE table=default.Table_341a1994d8db4935ae73da15c9261fc7 [id=76979641b41541478efab8529dd4a0ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.047057 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aea80e2e708c41b78dc5433faa34b0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.051708 2386 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:18.052822 2386 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:18.053542 2386 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af]). Expected version 2 got 0
I20260927 23:28:18.054571 2386 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1c87484b8b9742faa683f13d37308e58 (table default.Table_4fa6aec871b84978973cb579e195404b [id=2cd91cbdb5b14e16b847253971c8cb25]). Expected version 1 got 0
I20260927 23:28:18.055117 2386 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 9eb27269a590476f878ee52a2aefd141 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]). Expected version 1 got 0
I20260927 23:28:18.057207 3756 ts_manager.cc:194] Registered new tserver with Master: abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:28:18.059743 4037 sasl_common.cc:323] Time spent creating new SASL kudu client: real 0.316s user 0.000s sys 0.001s
I20260927 23:28:18.064356 2327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 315 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:28:18.072871 3762 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" instance_seqno: 1790551670327620) as {username='slave'} at 127.2.57.1:40507; Asking this server to re-register.
I20260927 23:28:18.086966 2938 tablet_service.cc:1467] Processing CreateTablet for tablet 0c25e9be61be4dd2841aa2c100b43580 (DEFAULT_TABLE table=default.Table_d72c914ddbb24ee8ba380dc1d49175f0 [id=d3d7aad68f32434888e5e676c37b5571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.089594 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 315 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:28:18.091675 2938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c25e9be61be4dd2841aa2c100b43580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.091638 3695 leader_election.cc:302] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150; no voters: 19a88ff0709d4266b933cc526577127f, bee21c28e51643189178a3c851672546
I20260927 23:28:18.089586 2391 catalog_manager.cc:5495] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:18.097168 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:18.098989 2391 catalog_manager.cc:5495] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:18.103935 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 9eb27269a590476f878ee52a2aefd141 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]). Expected version 1 got 0
I20260927 23:28:18.104718 2391 catalog_manager.cc:5495] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1). New cstate: current_term: 1 leader_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:18.112651 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1c87484b8b9742faa683f13d37308e58 (table default.Table_4fa6aec871b84978973cb579e195404b [id=2cd91cbdb5b14e16b847253971c8cb25]). Expected version 1 got 0
I20260927 23:28:18.113710 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
I20260927 23:28:18.113833 2644 tablet_service.cc:1467] Processing CreateTablet for tablet 2c99ea48bb3e461e9fc0f4d95052c663 (DEFAULT_TABLE table=default.Table_047c5f89196f446e86279aae36cf7f99 [id=2aecf7c487414df1985aa5f284224515]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.114478 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 0c25e9be61be4dd2841aa2c100b43580 (DEFAULT_TABLE table=default.Table_d72c914ddbb24ee8ba380dc1d49175f0 [id=d3d7aad68f32434888e5e676c37b5571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.114737 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af]). Expected version 2 got 0
I20260927 23:28:18.115696 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 6d0d8a1f277242588f57e4df95df5d5a (table default.Table_47fc9fc598ce4ca4bb4f94a523b39f74 [id=7b1bf5063eef401eb9de9b6f7aa40e4b]). Expected version 1 got 0
I20260927 23:28:18.115620 2651 tablet_service.cc:1467] Processing CreateTablet for tablet aea80e2e708c41b78dc5433faa34b0f5 (DEFAULT_TABLE table=default.Table_341a1994d8db4935ae73da15c9261fc7 [id=76979641b41541478efab8529dd4a0ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.115656 2652 tablet_service.cc:1467] Processing CreateTablet for tablet e328482c065f442cbc285584c6ffade4 (DEFAULT_TABLE table=default.Table_d0bc6f0546da41d1b7ce9d874ad0b416 [id=f0f6097616f742d2a6e32550bb46537c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.119408 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:18.120189 2387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 884b80a1d31349c98f75525416b3c527
I20260927 23:28:18.120893 2668 raft_consensus.cc:2454] T 49883297d2b046b99f2ec1343c276ff9 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:28:18.121646 2387 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 302) took 1128 ms. Trace:
I20260927 23:28:18.124410 2936 tablet_service.cc:1514] Processing DeleteTablet for tablet 884b80a1d31349c98f75525416b3c527 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:17 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:18.125111 3336 tablet_replica.cc:330] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:18.125933 3336 raft_consensus.cc:2229] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:18.126442 3336 raft_consensus.cc:2258] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:18.126436 2646 tablet_service.cc:1514] Processing DeleteTablet for tablet 884b80a1d31349c98f75525416b3c527 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:17 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:18.121939 2387 rpcz_store.cc:276] 0927 23:28:16.992931 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:17.698980 (+706049us) service_pool.cc:227] Handling call
0927 23:28:17.733939 (+ 34959us) spinlock_profiling.cc:243] Waited 34.2 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b3466 00007f8bb3bf4b34 00007f8bb3bf4ac2 00007f8bb3bf49f1 00007f8bb3bf496c 00007f8bb3bf3c22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:17.769538 (+ 35599us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:17.770050 (+ 512us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:28:17.770230 (+ 180us) catalog_manager.cc:2761] Locking tablets
0927 23:28:17.770952 (+ 722us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:28:17.797201 (+ 26249us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:28:17.840920 (+ 43719us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:18.120095 (+279175us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:18.121578 (+ 1483us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:17.772938 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:17.773377 (+ 439us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:17.773404 (+ 27us) write_op.cc:435] Acquired schema lock
0927 23:28:17.773415 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:17.773736 (+ 321us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:17.773788 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:17.773815 (+ 27us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:17.774127 (+ 312us) tablet.cc:699] Row locks acquired
0927 23:28:17.774141 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:17.774277 (+ 136us) write_op.cc:270] Start()
0927 23:28:17.774384 (+ 107us) write_op.cc:276] Timestamp: P: 1790551697774219 usec, L: 0
0927 23:28:17.774403 (+ 19us) op_driver.cc:331] REPLICATION: starting
0927 23:28:17.774949 (+ 546us) log.cc:842] Serialized 756 byte log entry
0927 23:28:17.786206 (+ 11257us) op_driver.cc:444] REPLICATION: finished
0927 23:28:17.786553 (+ 347us) write_op.cc:301] APPLY: starting
0927 23:28:17.786615 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:17.786747 (+ 132us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:17.786866 (+ 119us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:17.787141 (+ 275us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:17.787297 (+ 156us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:17.787342 (+ 45us) write_op.cc:312] APPLY: finished
0927 23:28:17.787549 (+ 207us) log.cc:842] Serialized 29 byte log entry
0927 23:28:17.787693 (+ 144us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.787781 (+ 88us) write_op.cc:454] Released schema lock
0927 23:28:17.787813 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:17.789442 (+ 1629us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.789467 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":78614400,"child_traces":[["op",{"apply.queue_time_us":185,"apply.run_cpu_time_us":2582,"apply.run_wall_time_us":3116,"num_ops":2,"prepare.queue_time_us":116,"prepare.run_cpu_time_us":2290,"prepare.run_wall_time_us":2290,"raft.queue_time_us":3211,"raft.run_cpu_time_us":687,"raft.run_wall_time_us":686,"replication_time_us":11714}]]}
I20260927 23:28:18.129325 2934 tablet_service.cc:1467] Processing CreateTablet for tablet e328482c065f442cbc285584c6ffade4 (DEFAULT_TABLE table=default.Table_d0bc6f0546da41d1b7ce9d874ad0b416 [id=f0f6097616f742d2a6e32550bb46537c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.129216 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_85a7076356114d53a7369e4ddda93a58"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:18.130769 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_85a7076356114d53a7369e4ddda93a58 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:18.130851 3336 ts_tablet_manager.cc:1905] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:18.133836 2392 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:51972 (request call id 298) took 2189 ms. Trace:
I20260927 23:28:18.134131 2392 rpcz_store.cc:276] 0927 23:28:15.944803 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:16.871155 (+926352us) service_pool.cc:227] Handling call
0927 23:28:16.872204 (+ 1049us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:17.024851 (+152647us) catalog_manager.cc:3616] Apply alter schema
0927 23:28:17.092432 (+ 67581us) catalog_manager.cc:3639] Acquired catalog manager lock
0927 23:28:17.092717 (+ 285us) catalog_manager.cc:3786] Updating metadata on disk
0927 23:28:17.683507 (+590790us) catalog_manager.cc:3817] Committing alterations to in-memory state
0927 23:28:18.133748 (+450241us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:17.523288 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:17.523784 (+ 496us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:17.523808 (+ 24us) write_op.cc:435] Acquired schema lock
0927 23:28:17.523820 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:17.524092 (+ 272us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:17.524133 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:17.524155 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:17.524322 (+ 167us) tablet.cc:699] Row locks acquired
0927 23:28:17.524340 (+ 18us) write_op.cc:260] PREPARE: finished
0927 23:28:17.524459 (+ 119us) write_op.cc:270] Start()
0927 23:28:17.573874 (+ 49415us) write_op.cc:276] Timestamp: P: 1790551697524375 usec, L: 0
0927 23:28:17.573895 (+ 21us) op_driver.cc:331] REPLICATION: starting
0927 23:28:17.574525 (+ 630us) log.cc:842] Serialized 405 byte log entry
0927 23:28:17.620839 (+ 46314us) op_driver.cc:444] REPLICATION: finished
0927 23:28:17.644942 (+ 24103us) write_op.cc:301] APPLY: starting
0927 23:28:17.646555 (+ 1613us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:17.647000 (+ 445us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:17.647573 (+ 573us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:17.650241 (+ 2668us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:17.682593 (+ 32352us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:17.682652 (+ 59us) write_op.cc:312] APPLY: finished
0927 23:28:17.682917 (+ 265us) log.cc:842] Serialized 23 byte log entry
0927 23:28:17.683127 (+ 210us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.683212 (+ 85us) write_op.cc:454] Released schema lock
0927 23:28:17.683249 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:17.683843 (+ 594us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.683871 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":23603,"apply.run_cpu_time_us":2227,"apply.run_wall_time_us":39305,"num_ops":1,"prepare.queue_time_us":429341,"prepare.run_cpu_time_us":2181,"prepare.run_wall_time_us":63092,"replication_time_us":46868,"thread_start_us":63212,"threads_started":1}]]}
I20260927 23:28:18.143198 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_a8a653897aa146dba5f2a3dba50fde33"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:18.145200 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a8a653897aa146dba5f2a3dba50fde33 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:18.146723 2936 tablet_service.cc:1467] Processing CreateTablet for tablet aa34927d61b04a00a4da78fb9537d9c7 (DEFAULT_TABLE table=default.Table_71487b69136749d4abdcb15306ae8daf [id=c191815c282a4d5582321e4fba40cc73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.147953 2936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa34927d61b04a00a4da78fb9537d9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.158635 3070 tablet_bootstrap.cc:492] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:18.159121 3070 ts_tablet_manager.cc:1397] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.435s user 0.010s sys 0.003s
I20260927 23:28:18.161767 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_b7548cd9beb941bd83aa4939fa976df2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:18.163545 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b7548cd9beb941bd83aa4939fa976df2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:18.165071 2934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e328482c065f442cbc285584c6ffade4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:18.166224 4053 log.cc:924] Time spent T 3a0578b6c99c4209a8d3c76f2089ef46 P 5ce671f028a34ea4b8f0520c2689c9a1: Append to log took a long time: real 0.199s user 0.000s sys 0.000s
W20260927 23:28:17.963688 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 46619.2us
I20260927 23:28:18.002594 3167 tablet_bootstrap.cc:492] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:18.179452 2794 tablet_service.cc:1467] Processing CreateTablet for tablet 909decfb6f0040bea61190fe44e9ae07 (DEFAULT_TABLE table=default.Table_f6b1317b23f04156946dfcd7dc25c076 [id=f66ef7897a0c4f8c85274caf777b2c76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.180738 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 909decfb6f0040bea61190fe44e9ae07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.003505 2802 tablet_service.cc:1467] Processing CreateTablet for tablet b9c86c7d874a49178c7b2cd7316afa14 (DEFAULT_TABLE table=default.Table_f45431c208e642ef9f406af42ff7d455 [id=d44851dff14d4e5084ae024995202140]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.010627 2804 tablet_service.cc:1467] Processing CreateTablet for tablet 2c99ea48bb3e461e9fc0f4d95052c663 (DEFAULT_TABLE table=default.Table_047c5f89196f446e86279aae36cf7f99 [id=2aecf7c487414df1985aa5f284224515]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.041589 2826 raft_consensus.cc:3037] T 5c87d497da884544920b3be9c78daccf P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:18.043139 2795 tablet_service.cc:1467] Processing CreateTablet for tablet aea80e2e708c41b78dc5433faa34b0f5 (DEFAULT_TABLE table=default.Table_341a1994d8db4935ae73da15c9261fc7 [id=76979641b41541478efab8529dd4a0ca]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:18.081027 2761 consensus_peers.cc:487] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 -> Peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Couldn't send request to peer de216a4a8e4b4e1b857d8647a3b05f6d. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbeeccc19a504e7984b293127766daf0. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:18.081557 2805 tablet_service.cc:1467] Processing CreateTablet for tablet 0c25e9be61be4dd2841aa2c100b43580 (DEFAULT_TABLE table=default.Table_d72c914ddbb24ee8ba380dc1d49175f0 [id=d3d7aad68f32434888e5e676c37b5571]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:18.182874 2761 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 102089us
I20260927 23:28:18.107590 2796 tablet_service.cc:1467] Processing CreateTablet for tablet e328482c065f442cbc285584c6ffade4 (DEFAULT_TABLE table=default.Table_d0bc6f0546da41d1b7ce9d874ad0b416 [id=f0f6097616f742d2a6e32550bb46537c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.126005 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet 884b80a1d31349c98f75525416b3c527 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:17 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:18.132563 2801 tablet_service.cc:1467] Processing CreateTablet for tablet aa34927d61b04a00a4da78fb9537d9c7 (DEFAULT_TABLE table=default.Table_71487b69136749d4abdcb15306ae8daf [id=c191815c282a4d5582321e4fba40cc73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.185352 4049 tablet_replica.cc:330] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:18.185664 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6bd8b531180f48d59cf8a90f3ea82149. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.186169 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c99ea48bb3e461e9fc0f4d95052c663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.186154 4049 raft_consensus.cc:2229] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:18.186933 2795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aea80e2e708c41b78dc5433faa34b0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.187366 4049 raft_consensus.cc:2258] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:18.189493 4049 ts_tablet_manager.cc:1905] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:18.189551 4004 consensus_queue.cc:1035] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:18.190433 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa34927d61b04a00a4da78fb9537d9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.191043 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c86c7d874a49178c7b2cd7316afa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.196805 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c25e9be61be4dd2841aa2c100b43580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.200314 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e328482c065f442cbc285584c6ffade4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.201321 3167 tablet_bootstrap.cc:654] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:18.213749 4058 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:28:18.214342 4005 raft_consensus.cc:686] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Becoming Leader. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Running, Role: LEADER
I20260927 23:28:18.218757 2392 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_9b9d6fbb353e4f3891265e29c4bd8778"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:18.222296 2392 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_9b9d6fbb353e4f3891265e29c4bd8778 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:18.240792 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 75584.5us
I20260927 23:28:18.242034 4001 consensus_queue.cc:1035] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:18.246402 2646 tablet_service.cc:1467] Processing CreateTablet for tablet aa34927d61b04a00a4da78fb9537d9c7 (DEFAULT_TABLE table=default.Table_71487b69136749d4abdcb15306ae8daf [id=c191815c282a4d5582321e4fba40cc73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.248593 4046 consensus_queue.cc:1035] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:18.226823 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 4eff8f3e5ef74e93b077b5e5bfb139c6 (table default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af]). Expected version 2 got 0
I20260927 23:28:18.250416 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:18.251509 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 1c87484b8b9742faa683f13d37308e58 (table default.Table_4fa6aec871b84978973cb579e195404b [id=2cd91cbdb5b14e16b847253971c8cb25]). Expected version 1 got 0
W20260927 23:28:18.252450 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 82394.1us
I20260927 23:28:18.252998 2645 tablet_service.cc:1467] Processing CreateTablet for tablet 909decfb6f0040bea61190fe44e9ae07 (DEFAULT_TABLE table=default.Table_f6b1317b23f04156946dfcd7dc25c076 [id=f66ef7897a0c4f8c85274caf777b2c76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.254137 4001 raft_consensus.cc:482] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:18.254565 4001 raft_consensus.cc:504] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.256556 4001 leader_election.cc:288] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:18.256991 4043 consensus_queue.cc:1035] T 9f4a9d69d75c4bfbb43e558b83b460d1 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:18.252231 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
I20260927 23:28:18.262125 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:18.262884 2390 catalog_manager.cc:5495] T 3a0578b6c99c4209a8d3c76f2089ef46 P de216a4a8e4b4e1b857d8647a3b05f6d reported cstate change: term changed from 0 to 1, leader changed from <none> to de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3). New cstate: current_term: 1 leader_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: HEALTHY } } }
I20260927 23:28:18.250978 2807 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea5e317ad5f4163b9d5bba8d4e85337 (DEFAULT_TABLE table=default.Table_1e1f242e1c804c9b883131486cbc0125 [id=187aa6b9c0594da2a279acd92cccb311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.274796 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea5e317ad5f4163b9d5bba8d4e85337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.271313 3167 tablet_bootstrap.cc:492] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:18.276079 3167 ts_tablet_manager.cc:1397] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.274s user 0.005s sys 0.006s
I20260927 23:28:18.276937 2390 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 9eb27269a590476f878ee52a2aefd141 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]). Expected version 1 got 0
I20260927 23:28:18.277567 4005 consensus_queue.cc:237] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.278420 3167 raft_consensus.cc:348] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.279064 3167 raft_consensus.cc:374] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:18.279325 3167 raft_consensus.cc:729] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:18.279692 2388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 9670df27c4404acda97661c69420b5cf
I20260927 23:28:18.281215 2388 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 297) took 2578 ms. Trace:
I20260927 23:28:18.281498 2388 rpcz_store.cc:276] 0927 23:28:15.702865 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:16.836426 (+1133561us) service_pool.cc:227] Handling call
0927 23:28:16.837864 (+ 1438us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:16.838377 (+ 513us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:28:16.838524 (+ 147us) catalog_manager.cc:2761] Locking tablets
0927 23:28:16.949694 (+111170us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:28:17.637021 (+687327us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:28:17.679050 (+ 42029us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:18.279580 (+600530us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:18.281114 (+ 1534us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:17.515939 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:17.516388 (+ 449us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:17.516420 (+ 32us) write_op.cc:435] Acquired schema lock
0927 23:28:17.516431 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:17.516813 (+ 382us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:17.516856 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:17.516881 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:17.517183 (+ 302us) tablet.cc:699] Row locks acquired
0927 23:28:17.517202 (+ 19us) write_op.cc:260] PREPARE: finished
0927 23:28:17.517329 (+ 127us) write_op.cc:270] Start()
0927 23:28:17.517424 (+ 95us) write_op.cc:276] Timestamp: P: 1790551697517246 usec, L: 0
0927 23:28:17.517440 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:17.518064 (+ 624us) log.cc:842] Serialized 762 byte log entry
0927 23:28:17.620420 (+102356us) op_driver.cc:444] REPLICATION: finished
0927 23:28:17.635309 (+ 14889us) write_op.cc:301] APPLY: starting
0927 23:28:17.635402 (+ 93us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:17.635526 (+ 124us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:17.635679 (+ 153us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:17.636006 (+ 327us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:17.636181 (+ 175us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:17.636229 (+ 48us) write_op.cc:312] APPLY: finished
0927 23:28:17.636486 (+ 257us) log.cc:842] Serialized 29 byte log entry
0927 23:28:17.636663 (+ 177us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.636747 (+ 84us) write_op.cc:454] Released schema lock
0927 23:28:17.636782 (+ 35us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:17.637231 (+ 449us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.637283 (+ 52us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":14073,"apply.run_cpu_time_us":2035,"apply.run_wall_time_us":2464,"num_ops":2,"prepare.queue_time_us":564029,"prepare.run_cpu_time_us":2286,"prepare.run_wall_time_us":3612,"replication_time_us":102831}]]}
I20260927 23:28:18.285565 2798 tablet_service.cc:1514] Processing DeleteTablet for tablet 9670df27c4404acda97661c69420b5cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:16 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:18.286499 2654 tablet_service.cc:1514] Processing DeleteTablet for tablet 9670df27c4404acda97661c69420b5cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:16 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:18.228890 3695 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.61:34987, user_credentials={real_user=slave}} blocked reactor thread for 137346us
I20260927 23:28:18.299302 2654 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea5e317ad5f4163b9d5bba8d4e85337 (DEFAULT_TABLE table=default.Table_1e1f242e1c804c9b883131486cbc0125 [id=187aa6b9c0594da2a279acd92cccb311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.308866 2721 heartbeater.cc:464] Registering TS with master...
I20260927 23:28:18.313369 2824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31676cfbbad04e728adb4b8aa8d905a6" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:18.280143 3167 consensus_queue.cc:260] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.318821 3336 ts_tablet_manager.cc:1918] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:18.319228 3336 log.cc:1196] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/884b80a1d31349c98f75525416b3c527
I20260927 23:28:18.319329 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 2c99ea48bb3e461e9fc0f4d95052c663 (DEFAULT_TABLE table=default.Table_047c5f89196f446e86279aae36cf7f99 [id=2aecf7c487414df1985aa5f284224515]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.320317 2943 tablet_service.cc:1467] Processing CreateTablet for tablet 909decfb6f0040bea61190fe44e9ae07 (DEFAULT_TABLE table=default.Table_f6b1317b23f04156946dfcd7dc25c076 [id=f66ef7897a0c4f8c85274caf777b2c76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.320597 3167 ts_tablet_manager.cc:1428] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20260927 23:28:18.321359 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 909decfb6f0040bea61190fe44e9ae07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.321810 3167 tablet_bootstrap.cc:492] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:18.325664 3070 raft_consensus.cc:348] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.327359 3167 tablet_bootstrap.cc:654] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:18.328858 4056 raft_consensus.cc:482] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:18.329380 4056 raft_consensus.cc:504] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.331863 2388 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_ccc7b44c45f24b7c9b58a89e4b99f139" } modify_external_catalogs: true
I20260927 23:28:18.334246 4056 leader_election.cc:288] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:18.336567 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a708a517b744496c83a3aa759b3f1081" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:18.340950 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:18.344967 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
I20260927 23:28:18.345835 2392 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_a3208426010f46cf820de6a4dbba0099" } new_table_name: "default.Table_bfbb08d7d57a4207be6d5525e3aaf64c" modify_external_catalogs: true
I20260927 23:28:18.347250 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1c87484b8b9742faa683f13d37308e58 (table default.Table_4fa6aec871b84978973cb579e195404b [id=2cd91cbdb5b14e16b847253971c8cb25]). Expected version 1 got 0
I20260927 23:28:18.348546 2668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cd0972f635d4979abf5fd30e9b02b1d" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:18.349236 2668 raft_consensus.cc:2454] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:18.350493 2932 tablet_service.cc:1514] Processing DeleteTablet for tablet 9670df27c4404acda97661c69420b5cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:16 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:18.349929 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 9eb27269a590476f878ee52a2aefd141 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]). Expected version 1 got 0
I20260927 23:28:18.354780 2389 catalog_manager.cc:5495] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe182b2afc64d908f692ef45eae1d51 (127.2.57.2). New cstate: current_term: 1 leader_uuid: "abe182b2afc64d908f692ef45eae1d51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:18.355057 4049 ts_tablet_manager.cc:1918] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:18.358196 4049 log.cc:1196] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/884b80a1d31349c98f75525416b3c527
I20260927 23:28:18.359655 4049 ts_tablet_manager.cc:1939] T 884b80a1d31349c98f75525416b3c527 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:18.360713 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 6d0d8a1f277242588f57e4df95df5d5a (table default.Table_47fc9fc598ce4ca4bb4f94a523b39f74 [id=7b1bf5063eef401eb9de9b6f7aa40e4b]). Expected version 1 got 0
I20260927 23:28:18.365435 3336 ts_tablet_manager.cc:1939] T 884b80a1d31349c98f75525416b3c527 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:18.365767 2389 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:18.370261 2932 tablet_service.cc:1467] Processing CreateTablet for tablet 9ea5e317ad5f4163b9d5bba8d4e85337 (DEFAULT_TABLE table=default.Table_1e1f242e1c804c9b883131486cbc0125 [id=187aa6b9c0594da2a279acd92cccb311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:18.370837 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 884b80a1d31349c98f75525416b3c527 (table default.Table_c632a6c33818436ca316f423eb2b3aa7 [id=52c6e4c418ed403792b4bcb9e4ff3458]) successfully deleted
I20260927 23:28:18.376600 2386 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_0a327a97ce744e0783fc675f39f6d2fc [id=931a9b7ab80b47268596521d6be6a458] alter complete (version 1)
W20260927 23:28:18.389699 2386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 50) took 1696 ms (client timeout 1000 ms). Trace:
I20260927 23:28:18.393987 3167 tablet_bootstrap.cc:492] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:18.405040 2802 tablet_service.cc:1514] Processing DeleteTablet for tablet 2411d7c53a154fafad9edda9fa75bb36 with delete_type TABLET_DATA_DELETED (replaced by tablet 73cfb41befe44e5189029abafe7e326d) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:18.406002 2801 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:18.407501 2795 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da38499af164d7fb5c1a8d65167ad24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:14 UTC) from {username='slave'} at 127.0.0.1:42492
W20260927 23:28:18.390272 2386 rpcz_store.cc:269] 0927 23:28:16.693158 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:16.820552 (+127394us) service_pool.cc:227] Handling call
0927 23:28:18.102698 (+1282146us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:18.102723 (+ 25us) write_op.cc:454] Released schema lock
0927 23:28:18.389604 (+286881us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:18.089892 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:18.090488 (+ 596us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:18.090520 (+ 32us) write_op.cc:435] Acquired schema lock
0927 23:28:18.090534 (+ 14us) tablet.cc:660] Decoding operations
0927 23:28:18.090914 (+ 380us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:18.090972 (+ 58us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:18.090994 (+ 22us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:18.091156 (+ 162us) tablet.cc:699] Row locks acquired
0927 23:28:18.091170 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:28:18.091310 (+ 140us) write_op.cc:270] Start()
0927 23:28:18.091412 (+ 102us) write_op.cc:276] Timestamp: P: 1790551698091213 usec, L: 0
0927 23:28:18.091428 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:28:18.092005 (+ 577us) log.cc:842] Serialized 356 byte log entry
0927 23:28:18.100545 (+ 8540us) op_driver.cc:444] REPLICATION: finished
0927 23:28:18.100903 (+ 358us) write_op.cc:301] APPLY: starting
0927 23:28:18.100971 (+ 68us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:18.101076 (+ 105us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:18.101187 (+ 111us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:18.101381 (+ 194us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:18.101528 (+ 147us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:18.101569 (+ 41us) write_op.cc:312] APPLY: finished
0927 23:28:18.101761 (+ 192us) log.cc:842] Serialized 23 byte log entry
0927 23:28:18.101905 (+ 144us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:18.101974 (+ 69us) write_op.cc:454] Released schema lock
0927 23:28:18.102005 (+ 31us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":196,"apply.run_cpu_time_us":1282,"apply.run_wall_time_us":1281,"num_ops":1,"prepare.queue_time_us":19067,"prepare.run_cpu_time_us":2486,"prepare.run_wall_time_us":2486,"raft.queue_time_us":228,"raft.run_cpu_time_us":651,"raft.run_wall_time_us":649,"replication_time_us":9023}]]}
I20260927 23:28:18.413118 2386 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_d1cbbe2de1e64a64bda8507e7ac034bc"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:18.417704 2386 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d1cbbe2de1e64a64bda8507e7ac034bc in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:18.419044 2393 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 49883297d2b046b99f2ec1343c276ff9 consensus state has no leader
I20260927 23:28:18.429893 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9c86c7d874a49178c7b2cd7316afa14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.432807 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fe36a27814640a1a783a324189d46e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.436949 2644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c99ea48bb3e461e9fc0f4d95052c663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.437166 4070 tablet.cc:1722] T e104d49543934e5db3b188e1db065cdc P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:18.439819 3070 raft_consensus.cc:374] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:18.440152 3070 raft_consensus.cc:729] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:18.441193 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c25e9be61be4dd2841aa2c100b43580. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.363675 2971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a708a517b744496c83a3aa759b3f1081" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:18.444823 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aea80e2e708c41b78dc5433faa34b0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.445355 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e328482c065f442cbc285584c6ffade4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.446519 4050 tablet_replica.cc:330] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:18.447338 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa34927d61b04a00a4da78fb9537d9c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.451723 2645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 909decfb6f0040bea61190fe44e9ae07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.455235 2654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea5e317ad5f4163b9d5bba8d4e85337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.446302 3070 consensus_queue.cc:260] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.456259 3167 ts_tablet_manager.cc:1397] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.135s user 0.009s sys 0.002s
W20260927 23:28:18.444324 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 81120.7us
I20260927 23:28:18.459038 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c99ea48bb3e461e9fc0f4d95052c663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.462102 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_5d8d3564c4d844699dde7932dc4cb4e4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:18.462689 2393 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:51972 (request call id 301) took 2112 ms. Trace:
I20260927 23:28:18.462973 2393 rpcz_store.cc:276] 0927 23:28:16.350745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:17.510699 (+1159954us) service_pool.cc:227] Handling call
0927 23:28:17.511852 (+ 1153us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:17.512595 (+ 743us) catalog_manager.cc:3616] Apply alter schema
0927 23:28:17.513277 (+ 682us) catalog_manager.cc:3639] Acquired catalog manager lock
0927 23:28:17.513480 (+ 203us) catalog_manager.cc:3786] Updating metadata on disk
0927 23:28:17.708276 (+194796us) catalog_manager.cc:3817] Committing alterations to in-memory state
0927 23:28:18.462611 (+754335us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:17.602646 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:17.603127 (+ 481us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:17.603152 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:28:17.603167 (+ 15us) tablet.cc:660] Decoding operations
0927 23:28:17.603473 (+ 306us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:17.603514 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:17.603538 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:17.603705 (+ 167us) tablet.cc:699] Row locks acquired
0927 23:28:17.603723 (+ 18us) write_op.cc:260] PREPARE: finished
0927 23:28:17.603852 (+ 129us) write_op.cc:270] Start()
0927 23:28:17.603953 (+ 101us) write_op.cc:276] Timestamp: P: 1790551697603760 usec, L: 0
0927 23:28:17.603974 (+ 21us) op_driver.cc:331] REPLICATION: starting
0927 23:28:17.604541 (+ 567us) log.cc:842] Serialized 405 byte log entry
0927 23:28:17.684780 (+ 80239us) op_driver.cc:444] REPLICATION: finished
0927 23:28:17.706837 (+ 22057us) write_op.cc:301] APPLY: starting
0927 23:28:17.706899 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:17.706996 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:17.707089 (+ 93us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:17.707259 (+ 170us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:17.707456 (+ 197us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:17.707500 (+ 44us) write_op.cc:312] APPLY: finished
0927 23:28:17.707698 (+ 198us) log.cc:842] Serialized 23 byte log entry
0927 23:28:17.707853 (+ 155us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.707912 (+ 59us) write_op.cc:454] Released schema lock
0927 23:28:17.707941 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:17.708245 (+ 304us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:17.708272 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"child_traces":[["op",{"apply.queue_time_us":21918,"apply.run_cpu_time_us":1452,"apply.run_wall_time_us":1604,"num_ops":1,"prepare.queue_time_us":87990,"prepare.run_cpu_time_us":2182,"prepare.run_wall_time_us":13367,"replication_time_us":80727}]]}
I20260927 23:28:18.470111 2761 leader_election.cc:302] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:18.471421 4056 raft_consensus.cc:2726] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:18.474658 3070 ts_tablet_manager.cc:1428] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.315s user 0.006s sys 0.003s
I20260927 23:28:18.480396 3167 raft_consensus.cc:348] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.490336 3070 tablet_bootstrap.cc:492] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:18.499588 2391 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_55dca2c970f045b6966759363c4a0816 [id=ce182a32bf8a42e8990fbafc47dedc14] alter complete (version 2)
I20260927 23:28:18.505371 3167 raft_consensus.cc:374] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:18.505677 3167 raft_consensus.cc:729] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:18.506345 3167 consensus_queue.cc:260] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.524284 2653 tablet_service.cc:1514] Processing DeleteTablet for tablet bbeeccc19a504e7984b293127766daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet e268bcb4b0614f37a4d9d96f8f40d1cb) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:18.527860 3167 ts_tablet_manager.cc:1428] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.070s user 0.006s sys 0.000s
I20260927 23:28:18.528760 2643 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da38499af164d7fb5c1a8d65167ad24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:14 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:18.528990 3167 tablet_bootstrap.cc:492] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
W20260927 23:28:18.537931 2390 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 33) took 964 ms (client timeout 1000 ms). Trace:
I20260927 23:28:18.542395 2643 tablet_service.cc:1514] Processing DeleteTablet for tablet a1890852164d492db3540a97847888fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:54604
W20260927 23:28:18.538631 2390 rpcz_store.cc:269] 0927 23:28:17.573225 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:17.695028 (+121803us) service_pool.cc:227] Handling call
0927 23:28:17.736699 (+ 41671us) spinlock_profiling.cc:243] Waited 37.5 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:18.537851 (+801152us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:18.306488 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:18.306970 (+ 482us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:18.307000 (+ 30us) write_op.cc:435] Acquired schema lock
0927 23:28:18.307015 (+ 15us) tablet.cc:660] Decoding operations
0927 23:28:18.307328 (+ 313us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:18.307371 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:18.307398 (+ 27us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:18.307560 (+ 162us) tablet.cc:699] Row locks acquired
0927 23:28:18.307577 (+ 17us) write_op.cc:260] PREPARE: finished
0927 23:28:18.307730 (+ 153us) write_op.cc:270] Start()
0927 23:28:18.307823 (+ 93us) write_op.cc:276] Timestamp: P: 1790551698307647 usec, L: 0
0927 23:28:18.307842 (+ 19us) op_driver.cc:331] REPLICATION: starting
0927 23:28:18.308475 (+ 633us) log.cc:842] Serialized 524 byte log entry
0927 23:28:18.327808 (+ 19333us) op_driver.cc:444] REPLICATION: finished
0927 23:28:18.328156 (+ 348us) write_op.cc:301] APPLY: starting
0927 23:28:18.328215 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:18.328334 (+ 119us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:18.328435 (+ 101us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:18.328637 (+ 202us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:18.328769 (+ 132us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:18.328811 (+ 42us) write_op.cc:312] APPLY: finished
0927 23:28:18.329019 (+ 208us) log.cc:842] Serialized 23 byte log entry
0927 23:28:18.329166 (+ 147us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:18.329244 (+ 78us) write_op.cc:454] Released schema lock
0927 23:28:18.329278 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:18.329466 (+ 188us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:18.329493 (+ 27us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":21,"spinlock_wait_cycles":86339456,"child_traces":[["op",{"apply.queue_time_us":190,"apply.run_cpu_time_us":1500,"apply.run_wall_time_us":1500,"num_ops":1,"prepare.queue_time_us":11109,"prepare.run_cpu_time_us":2303,"prepare.run_wall_time_us":2302,"raft.queue_time_us":1966,"raft.run_cpu_time_us":746,"raft.run_wall_time_us":747,"replication_time_us":19870}]]}
W20260927 23:28:18.556427 2609 leader_election.cc:341] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:18.565423 3167 tablet_bootstrap.cc:654] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:18.573006 2385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 36) took 2664 ms (client timeout 1000 ms). Trace:
W20260927 23:28:18.573575 2385 rpcz_store.cc:269] 0927 23:28:15.908500 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:16.149504 (+241004us) service_pool.cc:227] Handling call
0927 23:28:18.572925 (+2423421us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":28515,"reported_tablets":30}
I20260927 23:28:18.466425 2388 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_ccc7b44c45f24b7c9b58a89e4b99f139" } modify_external_catalogs: true
W20260927 23:28:18.557104 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 150068us
W20260927 23:28:18.581948 2391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 37) took 1322 ms (client timeout 1000 ms). Trace:
I20260927 23:28:18.588752 4050 raft_consensus.cc:2229] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:18.590525 4056 raft_consensus.cc:482] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:18.590874 3072 tablet_bootstrap.cc:492] T 50dd9944d3134ab9b23a6109fd7d6ee5 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:18.591418 3072 ts_tablet_manager.cc:1397] T 50dd9944d3134ab9b23a6109fd7d6ee5 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.699s user 0.037s sys 0.038s
I20260927 23:28:18.591153 4056 raft_consensus.cc:504] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.593364 4056 leader_election.cc:288] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:18.597836 2384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 32) took 2337 ms (client timeout 1000 ms). Trace:
I20260927 23:28:18.607528 3167 tablet_bootstrap.cc:492] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:18.614468 3167 ts_tablet_manager.cc:1397] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.086s user 0.010s sys 0.003s
I20260927 23:28:18.617470 4056 raft_consensus.cc:482] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:18.617983 4056 raft_consensus.cc:504] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.619849 4056 leader_election.cc:288] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:18.620705 2668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0578b6c99c4209a8d3c76f2089ef46" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:18.623277 2643 tablet_service.cc:1514] Processing DeleteTablet for tablet 2411d7c53a154fafad9edda9fa75bb36 with delete_type TABLET_DATA_DELETED (replaced by tablet 73cfb41befe44e5189029abafe7e326d) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:18.624717 3070 tablet_bootstrap.cc:654] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:18.632546 4050 raft_consensus.cc:2258] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:18.631999 3167 raft_consensus.cc:348] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.634060 3167 raft_consensus.cc:374] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:18.634338 3167 raft_consensus.cc:729] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:18.634955 3167 consensus_queue.cc:260] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.637393 4049 tablet_replica.cc:330] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:18.643939 4050 ts_tablet_manager.cc:1905] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:18.649335 2964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31676cfbbad04e728adb4b8aa8d905a6" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:18.619961 2933 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da38499af164d7fb5c1a8d65167ad24 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:14 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:18.627460 2937 tablet_service.cc:1514] Processing DeleteTablet for tablet a1890852164d492db3540a97847888fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:18.671128 4001 consensus_queue.cc:1035] T 9eb27269a590476f878ee52a2aefd141 P 5ce671f028a34ea4b8f0520c2689c9a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:18.672353 3072 raft_consensus.cc:348] T 50dd9944d3134ab9b23a6109fd7d6ee5 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.673141 3072 raft_consensus.cc:374] T 50dd9944d3134ab9b23a6109fd7d6ee5 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:18.673405 3072 raft_consensus.cc:729] T 50dd9944d3134ab9b23a6109fd7d6ee5 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:18.673038 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f58349edf0db40f98951e116fc2582b0" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:18.674119 3072 consensus_queue.cc:260] T 50dd9944d3134ab9b23a6109fd7d6ee5 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:18.674911 2763 leader_election.cc:341] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:18.677265 3167 ts_tablet_manager.cc:1428] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.056s user 0.006s sys 0.001s
I20260927 23:28:18.679608 3167 tablet_bootstrap.cc:492] T 8cd0972f635d4979abf5fd30e9b02b1d P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:18.682366 2932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ea5e317ad5f4163b9d5bba8d4e85337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:18.683507 3336 tablet_replica.cc:330] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:18.685825 3336 raft_consensus.cc:2229] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:18.686497 3336 raft_consensus.cc:2258] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:18.689908 3336 ts_tablet_manager.cc:1905] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:18.695425 2947 consensus_meta.cc:287] T b9c86c7d874a49178c7b2cd7316afa14 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent flushing consensus metadata: real 0.592s user 0.006s sys 0.004s
I20260927 23:28:18.700011 3167 tablet_bootstrap.cc:654] T 8cd0972f635d4979abf5fd30e9b02b1d P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:18.642903 4049 raft_consensus.cc:2229] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:18.756721 4049 raft_consensus.cc:2258] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:18.758574 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 884b80a1d31349c98f75525416b3c527 (table default.Table_c632a6c33818436ca316f423eb2b3aa7 [id=52c6e4c418ed403792b4bcb9e4ff3458]) successfully deleted
I20260927 23:28:18.758756 2390 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_b967cf863a6848919c6542cb6bbb1c70" } modify_external_catalogs: true
I20260927 23:28:18.759263 2390 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_b967cf863a6848919c6542cb6bbb1c70" } modify_external_catalogs: true
I20260927 23:28:18.761107 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_f5c7bbc6853d4806aca1ad9a751e1e4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:18.762768 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f5c7bbc6853d4806aca1ad9a751e1e4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:18.598207 2384 rpcz_store.cc:269] 0927 23:28:16.260864 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:16.799448 (+538584us) service_pool.cc:227] Handling call
0927 23:28:18.597754 (+1798306us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:18.027557 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:18.027962 (+ 405us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:18.027989 (+ 27us) write_op.cc:435] Acquired schema lock
0927 23:28:18.028005 (+ 16us) tablet.cc:660] Decoding operations
0927 23:28:18.028371 (+ 366us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:18.028423 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:18.028450 (+ 27us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:18.028791 (+ 341us) tablet.cc:699] Row locks acquired
0927 23:28:18.028809 (+ 18us) write_op.cc:260] PREPARE: finished
0927 23:28:18.028948 (+ 139us) write_op.cc:270] Start()
0927 23:28:18.029047 (+ 99us) write_op.cc:276] Timestamp: P: 1790551698028864 usec, L: 0
0927 23:28:18.029068 (+ 21us) op_driver.cc:331] REPLICATION: starting
0927 23:28:18.029677 (+ 609us) log.cc:842] Serialized 895 byte log entry
0927 23:28:18.038166 (+ 8489us) op_driver.cc:444] REPLICATION: finished
0927 23:28:18.038516 (+ 350us) write_op.cc:301] APPLY: starting
0927 23:28:18.038578 (+ 62us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:18.038691 (+ 113us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:18.038789 (+ 98us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:18.039064 (+ 275us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:18.039200 (+ 136us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:18.039243 (+ 43us) write_op.cc:312] APPLY: finished
0927 23:28:18.039432 (+ 189us) log.cc:842] Serialized 29 byte log entry
0927 23:28:18.039571 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:18.039644 (+ 73us) write_op.cc:454] Released schema lock
0927 23:28:18.039676 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:18.039864 (+ 188us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:18.039886 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":199,"apply.run_cpu_time_us":1556,"apply.run_wall_time_us":1555,"num_ops":2,"prepare.queue_time_us":112,"prepare.run_cpu_time_us":2434,"prepare.run_wall_time_us":2434,"raft.queue_time_us":185,"raft.run_cpu_time_us":703,"raft.run_wall_time_us":702,"replication_time_us":9008}]]}
I20260927 23:28:18.765270 2384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_8abcb4595c95442ba71077feec547d2c" } new_table_name: "default.Table_ce8fbbe18f864958b1d2d0ccac9abaff" modify_external_catalogs: true
I20260927 23:28:18.758478 3167 tablet_bootstrap.cc:492] T 8cd0972f635d4979abf5fd30e9b02b1d P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:18.767046 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_1e6962ace257421683c16954e0f6b0b8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:18.767675 3167 ts_tablet_manager.cc:1397] T 8cd0972f635d4979abf5fd30e9b02b1d P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.088s user 0.006s sys 0.007s
W20260927 23:28:18.768777 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_1e6962ace257421683c16954e0f6b0b8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:18.463819 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_5d8d3564c4d844699dde7932dc4cb4e4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:18.770313 4049 ts_tablet_manager.cc:1905] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:18.770817 3167 raft_consensus.cc:348] T 8cd0972f635d4979abf5fd30e9b02b1d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.771485 3167 raft_consensus.cc:374] T 8cd0972f635d4979abf5fd30e9b02b1d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:18.772128 3167 raft_consensus.cc:729] T 8cd0972f635d4979abf5fd30e9b02b1d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:18.772991 3167 consensus_queue.cc:260] T 8cd0972f635d4979abf5fd30e9b02b1d P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.724332 4080 tablet.cc:1722] T e104d49543934e5db3b188e1db065cdc P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:18.775766 3167 ts_tablet_manager.cc:1428] T 8cd0972f635d4979abf5fd30e9b02b1d P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.008s user 0.002s sys 0.003s
W20260927 23:28:18.777681 2609 consensus_peers.cc:487] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 -> Peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Couldn't send request to peer abe182b2afc64d908f692ef45eae1d51. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:18.778302 2907 leader_election.cc:302] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d; no voters: abe182b2afc64d908f692ef45eae1d51
I20260927 23:28:18.782977 4069 raft_consensus.cc:2781] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:18.783351 4069 raft_consensus.cc:482] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:18.783625 4069 raft_consensus.cc:3037] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:18.786185 3167 tablet_bootstrap.cc:492] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:18.748245 3072 ts_tablet_manager.cc:1428] T 50dd9944d3134ab9b23a6109fd7d6ee5 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.157s user 0.000s sys 0.011s
I20260927 23:28:18.755795 2962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0578b6c99c4209a8d3c76f2089ef46" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:18.756997 2963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f58349edf0db40f98951e116fc2582b0" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:28:18.793510 2936 consensus_meta.cc:287] T aa34927d61b04a00a4da78fb9537d9c7 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent flushing consensus metadata: real 0.629s user 0.008s sys 0.007s
W20260927 23:28:18.801062 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 188472us
W20260927 23:28:18.804503 2608 leader_election.cc:341] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:18.804942 2608 leader_election.cc:302] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:18.805574 4048 raft_consensus.cc:2726] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:18.813299 3072 tablet_bootstrap.cc:492] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:18.821537 3167 tablet_bootstrap.cc:654] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:18.837040 2761 leader_election.cc:341] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:18.837625 2761 leader_election.cc:302] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:18.838346 4056 raft_consensus.cc:2726] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:18.838681 4069 raft_consensus.cc:504] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:18.583339 2391 rpcz_store.cc:269] 0927 23:28:17.259595 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:17.392540 (+132945us) service_pool.cc:227] Handling call
0927 23:28:17.515789 (+123249us) spinlock_profiling.cc:243] Waited 57 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:18.581859 (+1066070us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:18.150410 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:18.150878 (+ 468us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:18.150903 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:28:18.150919 (+ 16us) tablet.cc:660] Decoding operations
0927 23:28:18.151402 (+ 483us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:18.151447 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:18.151473 (+ 26us) tablet.cc:683] Acquiring locks for 3 operations
0927 23:28:18.151756 (+ 283us) tablet.cc:699] Row locks acquired
0927 23:28:18.151769 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:28:18.151897 (+ 128us) write_op.cc:270] Start()
0927 23:28:18.151988 (+ 91us) write_op.cc:276] Timestamp: P: 1790551698151850 usec, L: 0
0927 23:28:18.152006 (+ 18us) op_driver.cc:331] REPLICATION: starting
0927 23:28:18.154644 (+ 2638us) log.cc:842] Serialized 1266 byte log entry
0927 23:28:18.170200 (+ 15556us) op_driver.cc:444] REPLICATION: finished
0927 23:28:18.170528 (+ 328us) write_op.cc:301] APPLY: starting
0927 23:28:18.170586 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:18.170728 (+ 142us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:18.170856 (+ 128us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:18.171325 (+ 469us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:18.171524 (+ 199us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0927 23:28:18.171580 (+ 56us) write_op.cc:312] APPLY: finished
0927 23:28:18.171827 (+ 247us) log.cc:842] Serialized 35 byte log entry
0927 23:28:18.171991 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:18.172073 (+ 82us) write_op.cc:454] Released schema lock
0927 23:28:18.172105 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:18.172286 (+ 181us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:18.172332 (+ 46us) write_op.cc:454] Released schema lock
Related trace 'op':
0927 23:28:18.393265 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:18.393749 (+ 484us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:18.393779 (+ 30us) write_op.cc:435] Acquired schema lock
0927 23:28:18.393792 (+ 13us) tablet.cc:660] Decoding operations
0927 23:28:18.394128 (+ 336us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:18.394181 (+ 53us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:18.394220 (+ 39us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:18.394398 (+ 178us) tablet.cc:699] Row locks acquired
0927 23:28:18.394416 (+ 18us) write_op.cc:260] PREPARE: finished
0927 23:28:18.394555 (+ 139us) write_op.cc:270] Start()
0927 23:28:18.394658 (+ 103us) write_op.cc:276] Timestamp: P: 1790551698394484 usec, L: 0
0927 23:28:18.394675 (+ 17us) op_driver.cc:331] REPLICATION: starting
0927 23:28:18.395344 (+ 669us) log.cc:842] Serialized 356 byte log entry
0927 23:28:18.478361 (+ 83017us) op_driver.cc:444] REPLICATION: finished
0927 23:28:18.479170 (+ 809us) write_op.cc:301] APPLY: starting
0927 23:28:18.479236 (+ 66us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:18.479345 (+ 109us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:18.479464 (+ 119us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:18.479673 (+ 209us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:18.479823 (+ 150us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:18.479867 (+ 44us) write_op.cc:312] APPLY: finished
0927 23:28:18.480079 (+ 212us) log.cc:842] Serialized 23 byte log entry
0927 23:28:18.480240 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:18.499112 (+ 18872us) write_op.cc:454] Released schema lock
0927 23:28:18.499166 (+ 54us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:18.518760 (+ 19594us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:18.518794 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":31,"spinlock_wait_cycles":131095424,"child_traces":[["op",{"apply.queue_time_us":184,"apply.run_cpu_time_us":2003,"apply.run_wall_time_us":2002,"mutex_wait_us":13989,"num_ops":3,"prepare.queue_time_us":14171,"prepare.run_cpu_time_us":2428,"prepare.run_wall_time_us":4494,"replication_time_us":18111}],["op",{"apply.queue_time_us":638,"apply.run_cpu_time_us":1774,"apply.run_wall_time_us":39844,"num_ops":1,"prepare.queue_time_us":104,"prepare.run_cpu_time_us":2408,"prepare.run_wall_time_us":2407,"raft.queue_time_us":5090,"raft.run_cpu_time_us":904,"raft.run_wall_time_us":905,"replication_time_us":83589}]]}
I20260927 23:28:18.850325 4049 ts_tablet_manager.cc:1918] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:18.855130 3070 tablet_bootstrap.cc:492] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:18.855569 3070 ts_tablet_manager.cc:1397] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.374s user 0.011s sys 0.003s
I20260927 23:28:18.856493 4049 log.cc:1196] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/9670df27c4404acda97661c69420b5cf
I20260927 23:28:18.858623 4049 ts_tablet_manager.cc:1939] T 9670df27c4404acda97661c69420b5cf P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:18.860558 3072 tablet_bootstrap.cc:654] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:18.869271 4069 leader_election.cc:288] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
W20260927 23:28:18.869619 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 65144.3us
I20260927 23:28:18.877264 4048 raft_consensus.cc:482] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:18.879582 4048 raft_consensus.cc:504] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.881770 4048 leader_election.cc:288] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:18.901505 4034 catalog_manager.cc:1259] Loaded cluster ID: 994f6841308149a98b641f5878f9d494
I20260927 23:28:18.901921 4034 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: loading cluster ID for follower catalog manager: success
I20260927 23:28:18.908591 4096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:28:18.839541 2761 leader_election.cc:302] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:18.911651 4096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:28:18.916563 4096 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987)
I20260927 23:28:18.918093 2327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 315 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:28:18.926759 4056 raft_consensus.cc:2726] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:18.928068 2761 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 88584.9us
W20260927 23:28:18.921260 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 69056.2us
I20260927 23:28:18.937866 3070 raft_consensus.cc:348] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.879348 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]) successfully deleted
W20260927 23:28:18.966095 2379 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 86771.2us
W20260927 23:28:18.967093 2379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 35892219ae7140e58c96707d08190d4d (table default.Table_2ed30e794ca8476ea9111e99393a0371 [id=8650cd807b99452ab96a59cea71c8e4c], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 35892219ae7140e58c96707d08190d4d consensus state has no leader
I20260927 23:28:18.968200 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 315 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
W20260927 23:28:18.968420 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 2411d7c53a154fafad9edda9fa75bb36 because the tablet was not found. No further retry: Not found: Tablet not found: 2411d7c53a154fafad9edda9fa75bb36
W20260927 23:28:18.969007 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 5bf856a05dff4fb4943fe8e1d9b7f385 because the tablet was not found. No further retry: Not found: Tablet not found: 5bf856a05dff4fb4943fe8e1d9b7f385
W20260927 23:28:18.969605 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 1da38499af164d7fb5c1a8d65167ad24 because the tablet was not found. No further retry: Not found: Tablet not found: 1da38499af164d7fb5c1a8d65167ad24
I20260927 23:28:18.970865 3695 leader_election.cc:302] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150; no voters: 19a88ff0709d4266b933cc526577127f, bee21c28e51643189178a3c851672546
I20260927 23:28:18.971884 4096 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:28:18.974797 4056 raft_consensus.cc:482] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:18.975342 4056 raft_consensus.cc:504] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:18.979120 4056 leader_election.cc:288] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:18.988663 2826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cd0972f635d4979abf5fd30e9b02b1d" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:18.989564 2826 raft_consensus.cc:3037] T 8cd0972f635d4979abf5fd30e9b02b1d P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:18.980387 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2f98dd905d40959dc676f94a4a595b" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
W20260927 23:28:18.997771 2763 leader_election.cc:341] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:18.998297 3070 raft_consensus.cc:374] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:18.998633 3070 raft_consensus.cc:729] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:18.999266 3070 consensus_queue.cc:260] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.004762 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db8996200d9c4c5f8572d1db2dbfe4e3" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:19.013779 3072 tablet_bootstrap.cc:492] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:19.013906 3167 tablet_bootstrap.cc:492] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:19.014330 3072 ts_tablet_manager.cc:1397] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.222s user 0.057s sys 0.073s
I20260927 23:28:19.014578 3167 ts_tablet_manager.cc:1397] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.229s user 0.050s sys 0.035s
I20260927 23:28:19.016993 3167 raft_consensus.cc:348] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.017052 3072 raft_consensus.cc:348] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.017719 3072 raft_consensus.cc:374] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:19.017983 3072 raft_consensus.cc:729] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:19.018610 3072 consensus_queue.cc:260] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.029664 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2f98dd905d40959dc676f94a4a595b" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:19.037725 3336 ts_tablet_manager.cc:1918] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:19.041646 4034 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: acquiring CA information for follower catalog manager: success
I20260927 23:28:19.048522 3336 log.cc:1196] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/9670df27c4404acda97661c69420b5cf
I20260927 23:28:19.056478 3336 ts_tablet_manager.cc:1939] T 9670df27c4404acda97661c69420b5cf P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
W20260927 23:28:19.060683 2392 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 216b4311431b44bca04b6a681b35537b consensus state has no leader
I20260927 23:28:19.061309 3167 raft_consensus.cc:374] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:19.061604 3167 raft_consensus.cc:729] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:19.062219 3167 consensus_queue.cc:260] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.026033 2826 raft_consensus.cc:2454] T 8cd0972f635d4979abf5fd30e9b02b1d P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:19.031587 4056 raft_consensus.cc:482] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:19.063300 4056 raft_consensus.cc:504] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.065632 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]) successfully deleted
I20260927 23:28:19.066565 3336 tablet_replica.cc:330] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:19.066545 4056 leader_election.cc:288] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:19.051101 2820 raft_consensus.cc:3037] T 3a0578b6c99c4209a8d3c76f2089ef46 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:19.052760 3762 ts_manager.cc:194] Registered new tserver with Master: 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:19.069000 2388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet a1a14a6135dc466881c1182450d7d4f0
I20260927 23:28:19.071467 3072 ts_tablet_manager.cc:1428] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.057s user 0.004s sys 0.002s
W20260927 23:28:19.057128 2761 leader_election.cc:341] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:19.072511 2761 leader_election.cc:302] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:19.077998 2934 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a14a6135dc466881c1182450d7d4f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:18 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:19.080611 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a14a6135dc466881c1182450d7d4f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:18 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:19.083825 2946 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:44952 (request call id 258) took 1043 ms. Trace:
W20260927 23:28:19.087321 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet a1a14a6135dc466881c1182450d7d4f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1a14a6135dc466881c1182450d7d4f0 already in progress: creating tablet
I20260927 23:28:19.089814 4050 ts_tablet_manager.cc:1918] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:19.090884 4056 raft_consensus.cc:2726] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:19.093425 2962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f253a54ab3b44822aee266314fa45495" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:28:19.100757 2761 leader_election.cc:341] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:19.100935 3072 tablet_bootstrap.cc:492] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:19.102125 3167 ts_tablet_manager.cc:1428] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.087s user 0.004s sys 0.000s
I20260927 23:28:19.103232 3167 tablet_bootstrap.cc:492] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:19.107514 4050 log.cc:1196] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/884b80a1d31349c98f75525416b3c527
I20260927 23:28:19.108767 4050 ts_tablet_manager.cc:1939] T 884b80a1d31349c98f75525416b3c527 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:19.110251 4050 tablet_replica.cc:330] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:19.084239 2946 rpcz_store.cc:276] 0927 23:28:18.040868 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:18.318142 (+277274us) service_pool.cc:227] Handling call
0927 23:28:18.322052 (+ 3910us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:18.322263 (+ 211us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:18.931123 (+608860us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:19.083247 (+152124us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:19.083751 (+ 504us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:19.113059 3336 raft_consensus.cc:2229] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:19.120010 3336 raft_consensus.cc:2258] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:19.120251 4056 raft_consensus.cc:482] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:19.120762 4056 raft_consensus.cc:504] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.121685 2388 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_d81346de05884027980a011d83c14bd2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:19.122745 4056 leader_election.cc:288] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:19.123379 2388 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_d81346de05884027980a011d83c14bd2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:19.123881 2962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50dd9944d3134ab9b23a6109fd7d6ee5" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:19.124553 2962 raft_consensus.cc:2454] T 50dd9944d3134ab9b23a6109fd7d6ee5 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:28:19.125610 2761 leader_election.cc:302] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:19.126328 4056 raft_consensus.cc:2781] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:19.126631 4056 raft_consensus.cc:482] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:19.126978 4056 raft_consensus.cc:3037] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:19.127975 3072 tablet_bootstrap.cc:654] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:19.137303 4050 raft_consensus.cc:2229] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:19.137470 3167 tablet_bootstrap.cc:654] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:19.122382 3336 ts_tablet_manager.cc:1905] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:19.151978 2384 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0d9370feeb454ad3850e3db76ea5519e consensus state has no leader
I20260927 23:28:19.163434 4056 raft_consensus.cc:504] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.165185 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db8996200d9c4c5f8572d1db2dbfe4e3" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:19.168979 2906 leader_election.cc:302] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:19.170231 3072 tablet_bootstrap.cc:492] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:19.170670 3072 ts_tablet_manager.cc:1397] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.070s user 0.009s sys 0.000s
I20260927 23:28:19.152055 4034 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 23:28:19.186426 4105 tablet.cc:1722] T e104d49543934e5db3b188e1db065cdc P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:19.218119 4068 raft_consensus.cc:2781] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:19.218668 4068 raft_consensus.cc:686] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:19.219556 4068 consensus_queue.cc:237] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.221364 3072 raft_consensus.cc:348] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.222183 3072 raft_consensus.cc:374] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:19.222429 3072 raft_consensus.cc:729] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:19.223117 3072 consensus_queue.cc:260] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.225827 3072 ts_tablet_manager.cc:1428] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.055s user 0.005s sys 0.002s
I20260927 23:28:19.226730 3072 tablet_bootstrap.cc:492] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:19.228190 3336 ts_tablet_manager.cc:1918] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:19.233395 3336 log.cc:1196] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/a1890852164d492db3540a97847888fd
I20260927 23:28:19.237149 3336 ts_tablet_manager.cc:1939] T a1890852164d492db3540a97847888fd P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
W20260927 23:28:19.228860 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 59970.9us
I20260927 23:28:19.232091 3072 tablet_bootstrap.cc:654] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:19.248603 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet a1890852164d492db3540a97847888fd (table default.Table_49c3f2d8a1b14c02878df5f1934bbaba [id=1b4c77aab36b477593fa78f19be095af]) successfully deleted
W20260927 23:28:19.259812 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 1da38499af164d7fb5c1a8d65167ad24 because the tablet was not found. No further retry: Not found: Tablet not found: 1da38499af164d7fb5c1a8d65167ad24
W20260927 23:28:19.260973 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet a1a14a6135dc466881c1182450d7d4f0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1a14a6135dc466881c1182450d7d4f0 already in progress: creating tablet
I20260927 23:28:19.262506 3167 tablet_bootstrap.cc:492] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:19.262130 2668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f253a54ab3b44822aee266314fa45495" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:19.262944 3167 ts_tablet_manager.cc:1397] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.160s user 0.017s sys 0.005s
I20260927 23:28:19.263185 2668 raft_consensus.cc:2454] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:28:19.266580 3070 ts_tablet_manager.cc:1428] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.411s user 0.004s sys 0.004s
I20260927 23:28:19.271651 2388 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_a902e331f690448a872f6c104857f2ca" } new_table_name: "default.Table_d0bd5ad8286d497792e879651b1fa58c" modify_external_catalogs: true
I20260927 23:28:19.277789 2393 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_85a7076356114d53a7369e4ddda93a58" } modify_external_catalogs: true
I20260927 23:28:19.278385 2393 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_85a7076356114d53a7369e4ddda93a58" } modify_external_catalogs: true
I20260927 23:28:19.286306 2386 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 9eb27269a590476f878ee52a2aefd141 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]). Expected version 1 got 0
I20260927 23:28:19.287320 2386 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 1c87484b8b9742faa683f13d37308e58 (table default.Table_4fa6aec871b84978973cb579e195404b [id=2cd91cbdb5b14e16b847253971c8cb25]). Expected version 1 got 0
I20260927 23:28:19.289376 2386 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 49883297d2b046b99f2ec1343c276ff9 (table default.Table_4458b554cada4d42bc1307905a783181 [id=adbf504950b64a08996f102f4ac2d662]). Expected version 1 got 0
I20260927 23:28:19.300370 3167 raft_consensus.cc:348] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.301091 3167 raft_consensus.cc:374] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:19.301328 3167 raft_consensus.cc:729] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:19.301903 3167 consensus_queue.cc:260] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.313066 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_4ea12f0a3fb449528dc3a1a406b707b6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:19.314791 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_4ea12f0a3fb449528dc3a1a406b707b6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:19.328100 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 65270668a2ee47f9b642ab2607b4ac59 (DEFAULT_TABLE table=default.Table_a8a653897aa146dba5f2a3dba50fde33 [id=3de9ac56e97b43fe9c62fb7cec9826f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.329329 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65270668a2ee47f9b642ab2607b4ac59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.333725 2807 tablet_service.cc:1467] Processing CreateTablet for tablet 65270668a2ee47f9b642ab2607b4ac59 (DEFAULT_TABLE table=default.Table_a8a653897aa146dba5f2a3dba50fde33 [id=3de9ac56e97b43fe9c62fb7cec9826f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.334842 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65270668a2ee47f9b642ab2607b4ac59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.338387 2934 tablet_service.cc:1467] Processing CreateTablet for tablet b545ed1dff2f4b70ae8f61165deb3fa4 (DEFAULT_TABLE table=default.Table_85a7076356114d53a7369e4ddda93a58 [id=6eb64141e9f74f2aa04b5cd7282bd9ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.339581 2934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b545ed1dff2f4b70ae8f61165deb3fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.341519 2633 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:54592 (request call id 32) took 1571 ms. Trace:
I20260927 23:28:19.341387 2763 leader_election.cc:302] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:19.341950 2633 rpcz_store.cc:276] 0927 23:28:17.770005 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:17.770120 (+ 115us) service_pool.cc:227] Handling call
0927 23:28:18.181593 (+411473us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b54000e15d4. stack: 00007fe1d7372033 00007fe1d7371f30 00007fe1d6ef3534 00000000004db2b4 00000000004db249 00007fe1de27e471 00007fe1de2f3268 00007fe1df2df6cd 00007fe1ddb87e04 00007fe1ddb87d92 00007fe1ddb87cc1 00007fe1ddb87c3c 00007fe1ddb86ef2 00007fe1dc0a894c 00007fe1dc0a7d86 00007fe1dc0a772f
0927 23:28:18.544722 (+363129us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b5c0005208c. stack: 00007fe1d7372033 00007fe1d7371f30 00007fe1d6ef3534 00000000004db2b4 00000000004db249 00007fe1de68e361 00007fe1de827fa9 00007fe1de71a22a 00007fe1df2df6a0 00007fe1ddb87e04 00007fe1ddb87d92 00007fe1ddb87cc1 00007fe1ddb87c3c 00007fe1ddb86ef2 00007fe1dc0a894c 00007fe1dc0a7d86
0927 23:28:19.341260 (+796538us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":318809344}
I20260927 23:28:19.342679 4005 raft_consensus.cc:2781] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:19.343014 4005 raft_consensus.cc:482] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:19.343441 4005 raft_consensus.cc:3037] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:19.345971 4050 raft_consensus.cc:2258] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:19.346252 3070 tablet_bootstrap.cc:492] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:19.358305 2384 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_2ca15ccb4f1f4b388143d9810dbb4ec6" } new_table_name: "default.Table_5770a6c3155b4700941fdaa33277b379" modify_external_catalogs: true
I20260927 23:28:19.358798 2392 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:19.359103 2826 raft_consensus.cc:3037] T a708a517b744496c83a3aa759b3f1081 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:19.359768 2392 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:19.360285 2392 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1c87484b8b9742faa683f13d37308e58 (table default.Table_4fa6aec871b84978973cb579e195404b [id=2cd91cbdb5b14e16b847253971c8cb25]). Expected version 1 got 0
I20260927 23:28:19.360914 2392 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 9eb27269a590476f878ee52a2aefd141 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]). Expected version 1 got 0
I20260927 23:28:19.378336 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50dd9944d3134ab9b23a6109fd7d6ee5" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:19.379345 2674 raft_consensus.cc:2454] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:28:19.380920 2650 tablet_service.cc:1514] Processing DeleteTablet for tablet a1a14a6135dc466881c1182450d7d4f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:18 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:19.382676 2652 tablet_service.cc:1467] Processing CreateTablet for tablet 65270668a2ee47f9b642ab2607b4ac59 (DEFAULT_TABLE table=default.Table_a8a653897aa146dba5f2a3dba50fde33 [id=3de9ac56e97b43fe9c62fb7cec9826f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.391326 2795 tablet_service.cc:1467] Processing CreateTablet for tablet b545ed1dff2f4b70ae8f61165deb3fa4 (DEFAULT_TABLE table=default.Table_85a7076356114d53a7369e4ddda93a58 [id=6eb64141e9f74f2aa04b5cd7282bd9ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.392678 2795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b545ed1dff2f4b70ae8f61165deb3fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.351954 4005 raft_consensus.cc:504] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.397903 2932 tablet_service.cc:1467] Processing CreateTablet for tablet e86b7f73fa704c4eae90ff752404b8c4 (DEFAULT_TABLE table=default.Table_8823e2b699aa4e13badf61047486415f [id=0292eee5ce5f45ebac4f8a0d817809bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.399220 2932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e86b7f73fa704c4eae90ff752404b8c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.405719 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65270668a2ee47f9b642ab2607b4ac59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.045518 2634 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:54592 (request call id 33) took 1142 ms. Trace:
I20260927 23:28:19.414048 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f253a54ab3b44822aee266314fa45495" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:19.414757 2672 raft_consensus.cc:3037] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:19.415383 2650 tablet_service.cc:1467] Processing CreateTablet for tablet b545ed1dff2f4b70ae8f61165deb3fa4 (DEFAULT_TABLE table=default.Table_85a7076356114d53a7369e4ddda93a58 [id=6eb64141e9f74f2aa04b5cd7282bd9ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.416765 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b545ed1dff2f4b70ae8f61165deb3fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.416990 4005 leader_election.cc:288] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:19.418084 4068 raft_consensus.cc:482] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:19.418565 4068 raft_consensus.cc:504] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.423254 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f253a54ab3b44822aee266314fa45495" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:19.424132 2827 raft_consensus.cc:2379] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate abe182b2afc64d908f692ef45eae1d51 in this term.
I20260927 23:28:19.426447 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8cd0972f635d4979abf5fd30e9b02b1d" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:19.427230 2673 raft_consensus.cc:3037] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:19.383719 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 884b80a1d31349c98f75525416b3c527 (table default.Table_c632a6c33818436ca316f423eb2b3aa7 [id=52c6e4c418ed403792b4bcb9e4ff3458]) successfully deleted
W20260927 23:28:19.432081 2380 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 48394.3us
I20260927 23:28:19.391922 2393 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet b545ed1dff2f4b70ae8f61165deb3fa4
I20260927 23:28:19.433674 2634 rpcz_store.cc:276] 0927 23:28:17.902982 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:17.903104 (+ 122us) service_pool.cc:227] Handling call
0927 23:28:18.696271 (+793167us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54000e0bd4. stack: 00007fe1d7372033 00007fe1d7371f30 00007fe1d6ef3534 00000000004db2b4 00000000004db249 00007fe1de27e471 00007fe1de2f3268 00007fe1df2df6cd 00007fe1ddb87e04 00007fe1ddb87d92 00007fe1ddb87cc1 00007fe1ddb87c3c 00007fe1ddb86ef2 00007fe1dc0a894c 00007fe1dc0a7d86 00007fe1dc0a772f
0927 23:28:19.045276 (+349005us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":644173184}
I20260927 23:28:19.434775 4068 leader_election.cc:288] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:19.435439 2393 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_b40fd6a9a57947d385d15e7eb107a5e1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:19.437041 2393 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_b40fd6a9a57947d385d15e7eb107a5e1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:19.398826 2387 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_164b8c9b1f7a4451a846dda60b831263"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:19.441330 2387 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_164b8c9b1f7a4451a846dda60b831263 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:19.444069 2673 raft_consensus.cc:2454] T 8cd0972f635d4979abf5fd30e9b02b1d P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:19.445593 2801 tablet_service.cc:1514] Processing DeleteTablet for tablet b545ed1dff2f4b70ae8f61165deb3fa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:19 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:19.449596 2938 tablet_service.cc:1514] Processing DeleteTablet for tablet b545ed1dff2f4b70ae8f61165deb3fa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:19 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:19.451093 2801 tablet_service.cc:1467] Processing CreateTablet for tablet e86b7f73fa704c4eae90ff752404b8c4 (DEFAULT_TABLE table=default.Table_8823e2b699aa4e13badf61047486415f [id=0292eee5ce5f45ebac4f8a0d817809bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.451254 2938 tablet_service.cc:1467] Processing CreateTablet for tablet 553730d5b95b42e5adbb6bf5091ccd3d (DEFAULT_TABLE table=default.Table_d656464366cb481aa6c28f4294bd5a99 [id=4e5a257192664a0388ab4d988fb4e064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.452235 2938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 553730d5b95b42e5adbb6bf5091ccd3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.452203 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e86b7f73fa704c4eae90ff752404b8c4. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:19.457441 2379 catalog_manager.cc:4808] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet b545ed1dff2f4b70ae8f61165deb3fa4 because the tablet was not found. No further retry: Not found: Tablet not found: b545ed1dff2f4b70ae8f61165deb3fa4
I20260927 23:28:19.459234 2802 tablet_service.cc:1467] Processing CreateTablet for tablet 553730d5b95b42e5adbb6bf5091ccd3d (DEFAULT_TABLE table=default.Table_d656464366cb481aa6c28f4294bd5a99 [id=4e5a257192664a0388ab4d988fb4e064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.459515 2796 tablet_service.cc:1467] Processing CreateTablet for tablet f0ea8d1a085b4104a046fe02f9f45724 (DEFAULT_TABLE table=default.Table_7340a0b8be8946d49f65fb36a2f6f09f [id=67238d27bdaf44cfab64015fd4d11620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.460372 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 553730d5b95b42e5adbb6bf5091ccd3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.460893 2796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0ea8d1a085b4104a046fe02f9f45724. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:19.462339 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet b545ed1dff2f4b70ae8f61165deb3fa4 because the tablet was not found. No further retry: Not found: Tablet not found: b545ed1dff2f4b70ae8f61165deb3fa4
I20260927 23:28:19.447016 2647 tablet_service.cc:1467] Processing CreateTablet for tablet e86b7f73fa704c4eae90ff752404b8c4 (DEFAULT_TABLE table=default.Table_8823e2b699aa4e13badf61047486415f [id=0292eee5ce5f45ebac4f8a0d817809bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.464980 2936 tablet_service.cc:1467] Processing CreateTablet for tablet f0ea8d1a085b4104a046fe02f9f45724 (DEFAULT_TABLE table=default.Table_7340a0b8be8946d49f65fb36a2f6f09f [id=67238d27bdaf44cfab64015fd4d11620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.466137 2936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0ea8d1a085b4104a046fe02f9f45724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.460372 2645 tablet_service.cc:1514] Processing DeleteTablet for tablet b545ed1dff2f4b70ae8f61165deb3fa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:19 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:19.468674 2651 tablet_service.cc:1467] Processing CreateTablet for tablet 553730d5b95b42e5adbb6bf5091ccd3d (DEFAULT_TABLE table=default.Table_d656464366cb481aa6c28f4294bd5a99 [id=4e5a257192664a0388ab4d988fb4e064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.469762 2647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e86b7f73fa704c4eae90ff752404b8c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.404042 2385 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_f92208741ec14cdbae4b59d76e210e05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:19.472808 3070 tablet_bootstrap.cc:654] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:19.473476 2385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_f92208741ec14cdbae4b59d76e210e05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:19.475558 2962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f253a54ab3b44822aee266314fa45495" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:19.476184 2962 raft_consensus.cc:3037] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:19.476262 2805 tablet_service.cc:1467] Processing CreateTablet for tablet d6c89691b3734d94a358fb394994d3d3 (DEFAULT_TABLE table=default.Table_f7dab7db47874af6b9466b544d9b8465 [id=79d1c5d616e4492baee2d600d4f77b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.477325 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c89691b3734d94a358fb394994d3d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.482827 2794 tablet_service.cc:1467] Processing CreateTablet for tablet 3560e3e8324a45078fd851d2dee445b9 (DEFAULT_TABLE table=default.Table_9b9d6fbb353e4f3891265e29c4bd8778 [id=d54756201bab4bb1ae72cf230611230a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.483990 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3560e3e8324a45078fd851d2dee445b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.489560 2804 tablet_service.cc:1467] Processing CreateTablet for tablet b02ba3c5171b4132ad1f3a9d41cf7ae8 (DEFAULT_TABLE table=default.Table_d31a1e740a2b42bd8569372845897579 [id=0240b7478a2140c0b03cbe472b1e8c27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.490653 2804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b02ba3c5171b4132ad1f3a9d41cf7ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.493541 2798 tablet_service.cc:1467] Processing CreateTablet for tablet 49509ebdaf1f4e3abf2d872733fbe959 (DEFAULT_TABLE table=default.Table_39a61687324b4fcbbd0ab963fed8c694 [id=1f9d20f6f4094b44833949ad65cca500]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.494673 2798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49509ebdaf1f4e3abf2d872733fbe959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.501101 2651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 553730d5b95b42e5adbb6bf5091ccd3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.501971 2947 tablet_service.cc:1467] Processing CreateTablet for tablet d6c89691b3734d94a358fb394994d3d3 (DEFAULT_TABLE table=default.Table_f7dab7db47874af6b9466b544d9b8465 [id=79d1c5d616e4492baee2d600d4f77b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.503059 2947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c89691b3734d94a358fb394994d3d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.506042 2943 tablet_service.cc:1467] Processing CreateTablet for tablet 3560e3e8324a45078fd851d2dee445b9 (DEFAULT_TABLE table=default.Table_9b9d6fbb353e4f3891265e29c4bd8778 [id=d54756201bab4bb1ae72cf230611230a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.507055 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3560e3e8324a45078fd851d2dee445b9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:19.513429 2609 leader_election.cc:341] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:19.525920 2645 tablet_service.cc:1467] Processing CreateTablet for tablet f0ea8d1a085b4104a046fe02f9f45724 (DEFAULT_TABLE table=default.Table_7340a0b8be8946d49f65fb36a2f6f09f [id=67238d27bdaf44cfab64015fd4d11620]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.526108 2648 tablet_service.cc:1467] Processing CreateTablet for tablet d6c89691b3734d94a358fb394994d3d3 (DEFAULT_TABLE table=default.Table_f7dab7db47874af6b9466b544d9b8465 [id=79d1c5d616e4492baee2d600d4f77b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.526522 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 3560e3e8324a45078fd851d2dee445b9 (DEFAULT_TABLE table=default.Table_9b9d6fbb353e4f3891265e29c4bd8778 [id=d54756201bab4bb1ae72cf230611230a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.526618 2933 tablet_service.cc:1467] Processing CreateTablet for tablet b02ba3c5171b4132ad1f3a9d41cf7ae8 (DEFAULT_TABLE table=default.Table_d31a1e740a2b42bd8569372845897579 [id=0240b7478a2140c0b03cbe472b1e8c27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.527717 2933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b02ba3c5171b4132ad1f3a9d41cf7ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.529296 2649 tablet_service.cc:1467] Processing CreateTablet for tablet b02ba3c5171b4132ad1f3a9d41cf7ae8 (DEFAULT_TABLE table=default.Table_d31a1e740a2b42bd8569372845897579 [id=0240b7478a2140c0b03cbe472b1e8c27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.531435 2645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0ea8d1a085b4104a046fe02f9f45724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.537070 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6c89691b3734d94a358fb394994d3d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.537982 2937 tablet_service.cc:1467] Processing CreateTablet for tablet 49509ebdaf1f4e3abf2d872733fbe959 (DEFAULT_TABLE table=default.Table_39a61687324b4fcbbd0ab963fed8c694 [id=1f9d20f6f4094b44833949ad65cca500]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.539115 2937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49509ebdaf1f4e3abf2d872733fbe959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.540913 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3560e3e8324a45078fd851d2dee445b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.544685 4068 raft_consensus.cc:482] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:19.544911 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b02ba3c5171b4132ad1f3a9d41cf7ae8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.545221 4068 raft_consensus.cc:504] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.547390 4068 leader_election.cc:288] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:19.549508 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f58349edf0db40f98951e116fc2582b0" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:19.550244 2827 raft_consensus.cc:2454] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:19.551478 2906 leader_election.cc:302] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:19.552351 4068 raft_consensus.cc:2781] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:19.552680 4068 raft_consensus.cc:482] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:19.553000 4068 raft_consensus.cc:3037] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:19.528174 4050 ts_tablet_manager.cc:1905] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:19.561115 4068 raft_consensus.cc:504] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.577915 2644 tablet_service.cc:1467] Processing CreateTablet for tablet 49509ebdaf1f4e3abf2d872733fbe959 (DEFAULT_TABLE table=default.Table_39a61687324b4fcbbd0ab963fed8c694 [id=1f9d20f6f4094b44833949ad65cca500]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:19.579061 2644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49509ebdaf1f4e3abf2d872733fbe959. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:19.583503 2827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f58349edf0db40f98951e116fc2582b0" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:19.584120 2827 raft_consensus.cc:3037] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:19.592633 2392 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 6d0d8a1f277242588f57e4df95df5d5a (table default.Table_47fc9fc598ce4ca4bb4f94a523b39f74 [id=7b1bf5063eef401eb9de9b6f7aa40e4b]). Expected version 1 got 0
I20260927 23:28:19.593551 2392 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
I20260927 23:28:19.594712 2392 catalog_manager.cc:5495] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe182b2afc64d908f692ef45eae1d51 (127.2.57.2). New cstate: current_term: 1 leader_uuid: "abe182b2afc64d908f692ef45eae1d51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:19.597078 4005 consensus_queue.cc:1035] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:19.612175 4068 leader_election.cc:288] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573)
I20260927 23:28:19.631282 2962 raft_consensus.cc:2454] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
W20260927 23:28:19.644444 2609 outbound_call.cc:322] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.2.57.62:38773, user_credentials={real_user=slave}} blocked reactor thread for 124704us
I20260927 23:28:19.673688 2672 raft_consensus.cc:2454] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:28:19.597599 2827 raft_consensus.cc:2454] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:19.697371 4112 consensus_queue.cc:1035] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:19.692368 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50dd9944d3134ab9b23a6109fd7d6ee5" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:19.709371 2673 raft_consensus.cc:3037] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:19.710254 2763 leader_election.cc:302] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
W20260927 23:28:19.699256 2608 leader_election.cc:341] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:19.714783 2608 leader_election.cc:302] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:19.717028 2673 raft_consensus.cc:2454] T 50dd9944d3134ab9b23a6109fd7d6ee5 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:28:19.718402 4046 raft_consensus.cc:2726] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:19.720239 2763 leader_election.cc:302] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters:
I20260927 23:28:19.721280 4112 raft_consensus.cc:2781] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:19.721747 4112 raft_consensus.cc:686] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Becoming Leader. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Running, Role: LEADER
I20260927 23:28:19.721897 4107 raft_consensus.cc:2781] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:19.722558 4112 consensus_queue.cc:237] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.653193 2390 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 73cfb41befe44e5189029abafe7e326d
I20260927 23:28:19.724965 2390 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 313) took 1232 ms. Trace:
I20260927 23:28:19.725288 2390 rpcz_store.cc:276] 0927 23:28:18.492768 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:18.551217 (+ 58449us) service_pool.cc:227] Handling call
0927 23:28:18.758645 (+207428us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b3466 00007f8bb3bf4b34 00007f8bb3bf4ac2 00007f8bb3bf49f1 00007f8bb3bf496c 00007f8bb3bf3c22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:18.759643 (+ 998us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:18.760141 (+ 498us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:28:18.760270 (+ 129us) catalog_manager.cc:2761] Locking tablets
0927 23:28:19.324974 (+564704us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:28:19.652738 (+327764us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:28:19.652977 (+ 239us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:19.653087 (+ 110us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:19.724897 (+ 71810us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:19.336217 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:19.336650 (+ 433us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:19.336675 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:28:19.336703 (+ 28us) tablet.cc:660] Decoding operations
0927 23:28:19.337060 (+ 357us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:19.337106 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:19.337131 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:19.337433 (+ 302us) tablet.cc:699] Row locks acquired
0927 23:28:19.337462 (+ 29us) write_op.cc:260] PREPARE: finished
0927 23:28:19.337595 (+ 133us) write_op.cc:270] Start()
0927 23:28:19.337682 (+ 87us) write_op.cc:276] Timestamp: P: 1790551699337534 usec, L: 0
0927 23:28:19.337801 (+ 119us) op_driver.cc:331] REPLICATION: starting
0927 23:28:19.338377 (+ 576us) log.cc:842] Serialized 722 byte log entry
0927 23:28:19.650929 (+312552us) op_driver.cc:444] REPLICATION: finished
0927 23:28:19.651244 (+ 315us) write_op.cc:301] APPLY: starting
0927 23:28:19.651305 (+ 61us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:19.651424 (+ 119us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:19.651521 (+ 97us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:19.651801 (+ 280us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:19.651947 (+ 146us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:19.651988 (+ 41us) write_op.cc:312] APPLY: finished
0927 23:28:19.652193 (+ 205us) log.cc:842] Serialized 29 byte log entry
0927 23:28:19.652401 (+ 208us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:19.652478 (+ 77us) write_op.cc:454] Released schema lock
0927 23:28:19.652511 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:19.653600 (+ 1089us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:19.653626 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":459531648,"child_traces":[["op",{"apply.queue_time_us":170,"apply.run_cpu_time_us":1713,"apply.run_wall_time_us":2579,"num_ops":2,"prepare.queue_time_us":9640,"prepare.run_cpu_time_us":2447,"prepare.run_wall_time_us":11239,"replication_time_us":313035}]]}
I20260927 23:28:19.729523 2651 tablet_service.cc:1514] Processing DeleteTablet for tablet 73cfb41befe44e5189029abafe7e326d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:18 UTC) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:19.733546 2798 tablet_service.cc:1514] Processing DeleteTablet for tablet 73cfb41befe44e5189029abafe7e326d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:18 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:19.702912 2393 master_service.cc:946] ReplaceTablet: received request to replace tablet fc60d0c7a3794607814393f033238052 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:19.734998 2392 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 49883297d2b046b99f2ec1343c276ff9 (table default.Table_4458b554cada4d42bc1307905a783181 [id=adbf504950b64a08996f102f4ac2d662]). Expected version 1 got 0
W20260927 23:28:19.735350 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 73cfb41befe44e5189029abafe7e326d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73cfb41befe44e5189029abafe7e326d already in progress: creating tablet
I20260927 23:28:19.694562 4056 leader_election.cc:288] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:19.743558 2384 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51972:
name: "default.Table_a6608c3398364885b077a2dfd090b14a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:19.745301 2384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.Table_a6608c3398364885b077a2dfd090b14a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:19.750268 2387 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_47fc9fc598ce4ca4bb4f94a523b39f74" } modify_external_catalogs: true
I20260927 23:28:19.751053 2387 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51972:
table { table_name: "default.Table_47fc9fc598ce4ca4bb4f94a523b39f74" } modify_external_catalogs: true
W20260927 23:28:19.768582 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 60244.8us
W20260927 23:28:19.779261 2388 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 0a2f98dd905d40959dc676f94a4a595b consensus state has no leader
I20260927 23:28:19.696286 2906 leader_election.cc:302] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
W20260927 23:28:19.790109 2906 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 93909.3us
I20260927 23:28:19.793215 4043 raft_consensus.cc:482] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:19.794083 2673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f58349edf0db40f98951e116fc2582b0" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:19.793694 4043 raft_consensus.cc:504] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.797168 4068 raft_consensus.cc:2781] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:19.797259 4043 leader_election.cc:288] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:19.731832 2936 tablet_service.cc:1514] Processing DeleteTablet for tablet 73cfb41befe44e5189029abafe7e326d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:18 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:19.804177 4068 raft_consensus.cc:686] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
W20260927 23:28:19.798280 2907 leader_election.cc:341] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:19.806048 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 73cfb41befe44e5189029abafe7e326d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 73cfb41befe44e5189029abafe7e326d already in progress: creating tablet
I20260927 23:28:19.805156 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f253a54ab3b44822aee266314fa45495" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:19.816531 2674 raft_consensus.cc:2379] T f253a54ab3b44822aee266314fa45495 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate de216a4a8e4b4e1b857d8647a3b05f6d in current term 1: Already voted for candidate abe182b2afc64d908f692ef45eae1d51 in this term.
I20260927 23:28:19.816900 4068 consensus_queue.cc:237] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.725261 2969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50dd9944d3134ab9b23a6109fd7d6ee5" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:19.822585 2969 raft_consensus.cc:3037] T 50dd9944d3134ab9b23a6109fd7d6ee5 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:19.828887 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f58349edf0db40f98951e116fc2582b0" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:19.831215 2968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee5d073c8df84f0884dc0f0d88eeed9f" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:19.848146 4128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:28:19.848834 4128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:28:19.852011 2327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 315 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:28:19.853792 3070 tablet_bootstrap.cc:492] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:19.854310 3070 ts_tablet_manager.cc:1397] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.509s user 0.011s sys 0.002s
I20260927 23:28:19.862637 4123 raft_consensus.cc:482] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:19.863330 4123 raft_consensus.cc:504] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.865619 4123 leader_election.cc:288] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:19.872974 2276 master-stress-test.cc:578] Killing test thread 1/10
I20260927 23:28:19.878754 2276 master-stress-test.cc:580] Killed test thread 1/10
I20260927 23:28:19.879040 2276 master-stress-test.cc:578] Killing test thread 2/10
I20260927 23:28:19.879272 2276 master-stress-test.cc:580] Killed test thread 2/10
I20260927 23:28:19.879451 2276 master-stress-test.cc:578] Killing test thread 3/10
I20260927 23:28:19.879650 2276 master-stress-test.cc:580] Killed test thread 3/10
I20260927 23:28:19.879833 2276 master-stress-test.cc:578] Killing test thread 4/10
I20260927 23:28:19.899785 2276 master-stress-test.cc:580] Killed test thread 4/10
I20260927 23:28:19.900127 2276 master-stress-test.cc:578] Killing test thread 5/10
I20260927 23:28:19.900424 2276 master-stress-test.cc:580] Killed test thread 5/10
I20260927 23:28:19.900624 2276 master-stress-test.cc:578] Killing test thread 6/10
I20260927 23:28:19.900846 2276 master-stress-test.cc:580] Killed test thread 6/10
I20260927 23:28:19.901033 2276 master-stress-test.cc:578] Killing test thread 7/10
W20260927 23:28:19.860859 3072 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.604s user 0.027s sys 0.033s
W20260927 23:28:19.909463 3072 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.653s user 0.028s sys 0.033s
I20260927 23:28:19.866140 4107 raft_consensus.cc:686] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Becoming Leader. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Running, Role: LEADER
W20260927 23:28:19.868371 2763 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 139767us
I20260927 23:28:19.913120 4107 consensus_queue.cc:237] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:19.833775 2608 leader_election.cc:341] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:19.920850 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 87116us
W20260927 23:28:19.921360 2761 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 77123us
I20260927 23:28:19.923866 3072 tablet_bootstrap.cc:492] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:19.928421 4128 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987)
I20260927 23:28:19.929348 3072 ts_tablet_manager.cc:1397] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.703s user 0.037s sys 0.034s
I20260927 23:28:19.929623 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 315 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:28:19.932767 3695 leader_election.cc:302] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150; no voters: 19a88ff0709d4266b933cc526577127f, bee21c28e51643189178a3c851672546
I20260927 23:28:19.934058 4128 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:28:19.940119 3072 raft_consensus.cc:348] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:19.940511 3167 ts_tablet_manager.cc:1428] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.671s user 0.006s sys 0.000s
I20260927 23:28:19.940764 3072 raft_consensus.cc:374] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260927 23:28:19.940775 3167 ts_tablet_manager.cc:1460] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51: Tablet startup took 671ms
I20260927 23:28:19.940977 3072 raft_consensus.cc:729] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:19.941529 3072 consensus_queue.cc:260] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:19.941004 3167 ts_tablet_manager.cc:1462] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51: Trace:
0927 23:28:08.516022 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:08.516369 (+ 347us) service_pool.cc:227] Handling call
0927 23:28:08.520034 (+ 3665us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:08.520100 (+ 66us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:08.529122 (+ 9022us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:08.577245 (+ 48123us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:08.577578 (+ 333us) inbound_call.cc:173] Queueing success response
0927 23:28:19.102908 (+10525330us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0927 23:28:19.269414 (+166506us) ts_tablet_manager.cc:1429] Starting tablet replica
0927 23:28:19.270025 (+ 611us) tablet_replica.cc:259] Starting instrumentation
0927 23:28:19.270554 (+ 529us) tablet_replica.cc:278] Starting consensus
Metrics: {"raft.queue_time_us":13935,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":169,"tablet-open.queue_time_us":10525374}
I20260927 23:28:19.946587 2969 raft_consensus.cc:2454] T 50dd9944d3134ab9b23a6109fd7d6ee5 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:28:19.967677 3072 ts_tablet_manager.cc:1428] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.038s user 0.004s sys 0.000s
I20260927 23:28:19.968566 3072 tablet_bootstrap.cc:492] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:19.992044 3072 tablet_bootstrap.cc:654] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:20.000449 2962 raft_consensus.cc:3037] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:20.010062 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:20.012631 2963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db8996200d9c4c5f8572d1db2dbfe4e3" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:28:20.013969 2379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 0a2f98dd905d40959dc676f94a4a595b (table default.Table_d0bd5ad8286d497792e879651b1fa58c [id=1acae7217ece44919a8d247f5cf155d1], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 0a2f98dd905d40959dc676f94a4a595b consensus state has no leader
I20260927 23:28:20.017613 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 9eb27269a590476f878ee52a2aefd141 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]). Expected version 1 got 0
I20260927 23:28:20.018400 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1c87484b8b9742faa683f13d37308e58 (table default.Table_4fa6aec871b84978973cb579e195404b [id=2cd91cbdb5b14e16b847253971c8cb25]). Expected version 1 got 0
I20260927 23:28:20.019062 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
I20260927 23:28:20.035384 4068 consensus_queue.cc:1035] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:20.019532 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:20.043362 2391 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 49883297d2b046b99f2ec1343c276ff9 (table default.Table_4458b554cada4d42bc1307905a783181 [id=adbf504950b64a08996f102f4ac2d662]). Expected version 1 got 0
I20260927 23:28:20.048230 3167 tablet_bootstrap.cc:492] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
W20260927 23:28:20.051858 2761 leader_election.cc:341] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:20.068202 2388 master_service.cc:946] ReplaceTablet: received request to replace tablet 4d5b59f73974430c91ba19c2ec307ea0 from {username='slave'} at 127.0.0.1:51972
I20260927 23:28:20.070813 3072 tablet_bootstrap.cc:492] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:20.071295 3072 ts_tablet_manager.cc:1397] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.103s user 0.008s sys 0.001s
I20260927 23:28:20.074776 3072 raft_consensus.cc:348] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.092519 3072 raft_consensus.cc:374] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:20.092897 3072 raft_consensus.cc:729] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:20.096547 3072 consensus_queue.cc:260] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.110136 4137 tablet.cc:1722] T 6ca42b47d8044adc99a82859bf79e606 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260927 23:28:20.105196 4105 tablet.cc:1722] T 6ca42b47d8044adc99a82859bf79e606 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260927 23:28:20.115418 3072 ts_tablet_manager.cc:1428] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.044s user 0.003s sys 0.002s
I20260927 23:28:20.116246 3072 tablet_bootstrap.cc:492] T ee5d073c8df84f0884dc0f0d88eeed9f P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:20.123917 3167 tablet_bootstrap.cc:654] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:20.130177 2907 leader_election.cc:302] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: de216a4a8e4b4e1b857d8647a3b05f6d; no voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51
W20260927 23:28:20.131208 2907 leader_election.cc:341] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:20.132776 4068 raft_consensus.cc:2726] T f253a54ab3b44822aee266314fa45495 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:20.173684 3072 tablet_bootstrap.cc:654] T ee5d073c8df84f0884dc0f0d88eeed9f P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:20.177686 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db8996200d9c4c5f8572d1db2dbfe4e3" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:20.178314 2672 raft_consensus.cc:2454] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
W20260927 23:28:20.106458 2282 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.ListTablets -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 46661us
I20260927 23:28:20.208276 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee5d073c8df84f0884dc0f0d88eeed9f" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:20.208634 3072 tablet_bootstrap.cc:492] T ee5d073c8df84f0884dc0f0d88eeed9f P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:20.209007 3072 ts_tablet_manager.cc:1397] T ee5d073c8df84f0884dc0f0d88eeed9f P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.093s user 0.009s sys 0.000s
I20260927 23:28:20.211047 3072 raft_consensus.cc:348] T ee5d073c8df84f0884dc0f0d88eeed9f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.211656 3072 raft_consensus.cc:374] T ee5d073c8df84f0884dc0f0d88eeed9f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:20.211867 3072 raft_consensus.cc:729] T ee5d073c8df84f0884dc0f0d88eeed9f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
W20260927 23:28:20.213050 2646 consensus_meta.cc:287] T 3560e3e8324a45078fd851d2dee445b9 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent flushing consensus metadata: real 0.520s user 0.007s sys 0.000s
I20260927 23:28:20.216939 2763 leader_election.cc:302] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:20.218914 4125 raft_consensus.cc:2781] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:20.219218 4125 raft_consensus.cc:482] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:20.219488 4125 raft_consensus.cc:3037] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:20.228871 4125 raft_consensus.cc:504] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.237994 4125 leader_election.cc:288] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:20.237272 3072 consensus_queue.cc:260] T ee5d073c8df84f0884dc0f0d88eeed9f P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.251878 2964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db8996200d9c4c5f8572d1db2dbfe4e3" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
W20260927 23:28:20.253145 2761 leader_election.cc:341] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:20.266728 3072 ts_tablet_manager.cc:1428] T ee5d073c8df84f0884dc0f0d88eeed9f P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.057s user 0.006s sys 0.000s
I20260927 23:28:20.267665 3072 tablet_bootstrap.cc:492] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:20.275844 2668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db8996200d9c4c5f8572d1db2dbfe4e3" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:20.276466 2668 raft_consensus.cc:3037] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:20.281957 3167 tablet_bootstrap.cc:492] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:20.282385 3167 ts_tablet_manager.cc:1397] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.234s user 0.019s sys 0.009s
I20260927 23:28:20.247910 3070 raft_consensus.cc:348] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.282953 3070 raft_consensus.cc:374] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:20.283227 3070 raft_consensus.cc:729] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ce671f028a34ea4b8f0520c2689c9a1, State: Initialized, Role: FOLLOWER
I20260927 23:28:20.283785 3070 consensus_queue.cc:260] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.284826 4107 raft_consensus.cc:482] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:20.285209 4107 raft_consensus.cc:504] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.286902 4107 leader_election.cc:288] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:20.288767 3072 tablet_bootstrap.cc:654] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:20.290347 4137 tablet.cc:1722] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:20.290691 4068 raft_consensus.cc:482] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:20.291090 4068 raft_consensus.cc:504] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.294574 4105 tablet.cc:1722] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:20.302733 3167 raft_consensus.cc:348] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.303417 3167 raft_consensus.cc:374] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:20.303643 3167 raft_consensus.cc:729] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:20.304167 3167 consensus_queue.cc:260] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.309849 2962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31676cfbbad04e728adb4b8aa8d905a6" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:28:20.312587 2761 leader_election.cc:341] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:20.313591 3070 ts_tablet_manager.cc:1428] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent starting tablet: real 0.459s user 0.007s sys 0.000s
I20260927 23:28:20.314477 3070 tablet_bootstrap.cc:492] T f58349edf0db40f98951e116fc2582b0 P 5ce671f028a34ea4b8f0520c2689c9a1: Bootstrap starting.
I20260927 23:28:20.319845 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2f98dd905d40959dc676f94a4a595b" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:20.320461 2820 raft_consensus.cc:2454] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:20.324970 2906 leader_election.cc:302] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:20.325667 4081 raft_consensus.cc:2781] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:20.325964 4081 raft_consensus.cc:482] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:20.326201 3167 ts_tablet_manager.cc:1428] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.044s user 0.007s sys 0.000s
I20260927 23:28:20.326233 4081 raft_consensus.cc:3037] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:20.327133 3167 tablet_bootstrap.cc:492] T 119a1338b04740d0bbf291c8fbe34992 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:20.341599 4068 leader_election.cc:288] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:20.352730 4081 raft_consensus.cc:504] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.356175 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2f98dd905d40959dc676f94a4a595b" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:20.356758 2820 raft_consensus.cc:3037] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:20.357645 2388 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 4d5b59f73974430c91ba19c2ec307ea0
I20260927 23:28:20.358027 3167 tablet_bootstrap.cc:654] T 119a1338b04740d0bbf291c8fbe34992 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:20.359488 2388 catalog_manager.cc:6565] ReplaceTablet: tablet 4d5b59f73974430c91ba19c2ec307ea0 deleted and replaced by tablet d8a335cd96dc4ae4af4207049495327f
I20260927 23:28:20.363972 2807 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d5b59f73974430c91ba19c2ec307ea0 with delete_type TABLET_DATA_DELETED (replaced by tablet d8a335cd96dc4ae4af4207049495327f) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:20.369693 2384 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 9eb27269a590476f878ee52a2aefd141 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]). Expected version 1 got 0
I20260927 23:28:20.370841 2384 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
I20260927 23:28:20.371368 2384 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 1c87484b8b9742faa683f13d37308e58 (table default.Table_4fa6aec871b84978973cb579e195404b [id=2cd91cbdb5b14e16b847253971c8cb25]). Expected version 1 got 0
I20260927 23:28:20.371827 2384 catalog_manager.cc:5587] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209) does not have the latest schema for tablet 49883297d2b046b99f2ec1343c276ff9 (table default.Table_4458b554cada4d42bc1307905a783181 [id=adbf504950b64a08996f102f4ac2d662]). Expected version 1 got 0
I20260927 23:28:20.382504 2934 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d5b59f73974430c91ba19c2ec307ea0 with delete_type TABLET_DATA_DELETED (replaced by tablet d8a335cd96dc4ae4af4207049495327f) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:20.384485 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:20.386881 2946 tablet_service.cc:1467] Processing CreateTablet for tablet 24900c5a9d154a90ac592034a86d0a37 (DEFAULT_TABLE table=default.Table_4ea12f0a3fb449528dc3a1a406b707b6 [id=7858e7c0328b4fd5ae53ce8427ca5305]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.387228 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 1c87484b8b9742faa683f13d37308e58 (table default.Table_4fa6aec871b84978973cb579e195404b [id=2cd91cbdb5b14e16b847253971c8cb25]). Expected version 1 got 0
I20260927 23:28:20.395067 3072 tablet_bootstrap.cc:492] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:20.395464 3072 ts_tablet_manager.cc:1397] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.128s user 0.011s sys 0.000s
I20260927 23:28:20.400658 4050 ts_tablet_manager.cc:1918] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:20.401311 4081 leader_election.cc:288] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
W20260927 23:28:20.406200 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 216802us
I20260927 23:28:20.411435 2807 tablet_service.cc:1467] Processing CreateTablet for tablet 24900c5a9d154a90ac592034a86d0a37 (DEFAULT_TABLE table=default.Table_4ea12f0a3fb449528dc3a1a406b707b6 [id=7858e7c0328b4fd5ae53ce8427ca5305]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.412462 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24900c5a9d154a90ac592034a86d0a37. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:20.413197 2379 catalog_manager.cc:4821] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): delete failed for tablet 4d5b59f73974430c91ba19c2ec307ea0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d5b59f73974430c91ba19c2ec307ea0 already in progress: creating tablet
I20260927 23:28:20.363943 2820 raft_consensus.cc:2454] T 0a2f98dd905d40959dc676f94a4a595b P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:20.417791 2906 leader_election.cc:302] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:20.418579 4081 raft_consensus.cc:2781] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
W20260927 23:28:20.418869 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 240528us
I20260927 23:28:20.409834 3072 raft_consensus.cc:348] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.419309 4081 raft_consensus.cc:686] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:20.420429 4081 consensus_queue.cc:237] T 0a2f98dd905d40959dc676f94a4a595b P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.428445 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:20.419605 3072 raft_consensus.cc:374] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:20.422621 2946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24900c5a9d154a90ac592034a86d0a37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.437077 2934 tablet_service.cc:1467] Processing CreateTablet for tablet 8e0e136fd83240369a5173b6b077ab9f (DEFAULT_TABLE table=default.Table_f5c7bbc6853d4806aca1ad9a751e1e4b [id=6b7de9bcff23494eba44920db81a9f4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.436179 3167 tablet_bootstrap.cc:492] T 119a1338b04740d0bbf291c8fbe34992 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:20.438123 3167 ts_tablet_manager.cc:1397] T 119a1338b04740d0bbf291c8fbe34992 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.111s user 0.008s sys 0.001s
I20260927 23:28:20.440279 3167 raft_consensus.cc:348] T 119a1338b04740d0bbf291c8fbe34992 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.441077 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 9eb27269a590476f878ee52a2aefd141 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]). Expected version 1 got 0
I20260927 23:28:20.440383 3072 raft_consensus.cc:729] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:20.445031 3072 consensus_queue.cc:260] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.402689 2668 raft_consensus.cc:2454] T db8996200d9c4c5f8572d1db2dbfe4e3 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
W20260927 23:28:20.447372 2349 debug-util.cc:398] Leaking SignalData structure 0x7b08000c3940 after lost signal to thread 2286
W20260927 23:28:20.448076 2349 debug-util.cc:398] Leaking SignalData structure 0x7b08000a25a0 after lost signal to thread 2352
I20260927 23:28:20.402825 4050 log.cc:1196] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/9670df27c4404acda97661c69420b5cf
I20260927 23:28:20.450803 4001 raft_consensus.cc:482] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:20.450889 4111 consensus_queue.cc:1035] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:20.451197 4001 raft_consensus.cc:504] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.442093 2802 tablet_service.cc:1467] Processing CreateTablet for tablet 8e0e136fd83240369a5173b6b077ab9f (DEFAULT_TABLE table=default.Table_f5c7bbc6853d4806aca1ad9a751e1e4b [id=6b7de9bcff23494eba44920db81a9f4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.452688 3167 raft_consensus.cc:374] T 119a1338b04740d0bbf291c8fbe34992 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:20.452958 3167 raft_consensus.cc:729] T 119a1338b04740d0bbf291c8fbe34992 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:20.453097 4001 leader_election.cc:288] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:20.454566 2802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e0e136fd83240369a5173b6b077ab9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.466621 3070 tablet_bootstrap.cc:654] T f58349edf0db40f98951e116fc2582b0 P 5ce671f028a34ea4b8f0520c2689c9a1: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:20.465003 2385 catalog_manager.cc:5495] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe182b2afc64d908f692ef45eae1d51 (127.2.57.2). New cstate: current_term: 1 leader_uuid: "abe182b2afc64d908f692ef45eae1d51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:20.467531 2672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2f98dd905d40959dc676f94a4a595b" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:20.468101 2672 raft_consensus.cc:2454] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:20.468808 3167 consensus_queue.cc:260] T 119a1338b04740d0bbf291c8fbe34992 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.470221 2932 tablet_service.cc:1467] Processing CreateTablet for tablet 2e045145d4ee4654b987f6101bd178ad (DEFAULT_TABLE table=default.Table_b7548cd9beb941bd83aa4939fa976df2 [id=7216d324b57e4efca64d3e60c6318660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.470670 2667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31676cfbbad04e728adb4b8aa8d905a6" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:20.471128 3167 ts_tablet_manager.cc:1428] T 119a1338b04740d0bbf291c8fbe34992 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.033s user 0.001s sys 0.004s
I20260927 23:28:20.471267 2667 raft_consensus.cc:2454] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:28:20.472378 2763 leader_election.cc:302] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:20.473489 2668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2f98dd905d40959dc676f94a4a595b" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:20.473515 2763 leader_election.cc:302] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1, abe182b2afc64d908f692ef45eae1d51; no voters: de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:20.473974 2668 raft_consensus.cc:3037] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:20.474925 2801 tablet_service.cc:1467] Processing CreateTablet for tablet 2e045145d4ee4654b987f6101bd178ad (DEFAULT_TABLE table=default.Table_b7548cd9beb941bd83aa4939fa976df2 [id=7216d324b57e4efca64d3e60c6318660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.475871 2801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e045145d4ee4654b987f6101bd178ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.477095 2938 tablet_service.cc:1467] Processing CreateTablet for tablet 561e72d79f1e405c993d0145a892ce72 (DEFAULT_TABLE table=default.Table_d81346de05884027980a011d83c14bd2 [id=2988c6c13fc240c3973738ad329ad5a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.477828 4107 raft_consensus.cc:2781] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:20.478125 4107 raft_consensus.cc:482] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:20.478437 4107 raft_consensus.cc:3037] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:20.479367 4050 ts_tablet_manager.cc:1939] T 9670df27c4404acda97661c69420b5cf P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting consensus metadata
I20260927 23:28:20.480656 4123 raft_consensus.cc:2781] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:20.481067 4123 raft_consensus.cc:686] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Becoming Leader. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Running, Role: LEADER
I20260927 23:28:20.481629 4050 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:54604 (request call id 294) took 2195 ms. Trace:
I20260927 23:28:20.481773 4123 consensus_queue.cc:237] T db8996200d9c4c5f8572d1db2dbfe4e3 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.481945 4050 rpcz_store.cc:276] 0927 23:28:18.286145 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:18.286313 (+ 168us) service_pool.cc:227] Handling call
0927 23:28:19.110076 (+823763us) ts_tablet_manager.cc:1145] Deleting tablet 9670df27c4404acda97661c69420b5cf
0927 23:28:19.110120 (+ 44us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:28:20.215688 (+1105568us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:20.400428 (+184740us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:20.481566 (+ 81138us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":217319,"tablet-delete.queue_time_us":823101}
I20260927 23:28:20.484221 4050 tablet_replica.cc:330] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1: stopping tablet replica
I20260927 23:28:20.485023 4050 raft_consensus.cc:2229] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:20.485921 2795 tablet_service.cc:1467] Processing CreateTablet for tablet 561e72d79f1e405c993d0145a892ce72 (DEFAULT_TABLE table=default.Table_d81346de05884027980a011d83c14bd2 [id=2988c6c13fc240c3973738ad329ad5a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.486968 2795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 561e72d79f1e405c993d0145a892ce72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.488709 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
I20260927 23:28:20.489301 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 49883297d2b046b99f2ec1343c276ff9 (table default.Table_4458b554cada4d42bc1307905a783181 [id=adbf504950b64a08996f102f4ac2d662]). Expected version 1 got 0
I20260927 23:28:20.489668 2933 tablet_service.cc:1467] Processing CreateTablet for tablet b72b96808c1c47909b468f94f68bfef1 (DEFAULT_TABLE table=default.Table_5d8d3564c4d844699dde7932dc4cb4e4 [id=cfe3ab6933794a50844075be6951c092]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.490321 2385 catalog_manager.cc:5495] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 reported cstate change: term changed from 0 to 1, leader changed from <none> to abe182b2afc64d908f692ef45eae1d51 (127.2.57.2). New cstate: current_term: 1 leader_uuid: "abe182b2afc64d908f692ef45eae1d51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } health_report { overall_health: UNKNOWN } } }
I20260927 23:28:20.491880 2385 catalog_manager.cc:5587] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573) does not have the latest schema for tablet 0a2f98dd905d40959dc676f94a4a595b (table default.Table_d0bd5ad8286d497792e879651b1fa58c [id=1acae7217ece44919a8d247f5cf155d1]). Expected version 1 got 0
I20260927 23:28:20.495303 4068 raft_consensus.cc:482] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:20.495718 4068 raft_consensus.cc:504] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.498371 4068 leader_election.cc:288] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:20.510383 2962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a2f98dd905d40959dc676f94a4a595b" candidate_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:20.516757 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119a1338b04740d0bbf291c8fbe34992" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51" is_pre_election: true
I20260927 23:28:20.517369 2820 raft_consensus.cc:2454] T 119a1338b04740d0bbf291c8fbe34992 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:20.518297 2906 leader_election.cc:302] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:20.519174 4068 raft_consensus.cc:2781] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:20.519357 4050 raft_consensus.cc:2258] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:20.519500 4068 raft_consensus.cc:482] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:20.519783 4068 raft_consensus.cc:3037] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:20.521503 4050 ts_tablet_manager.cc:1905] T bbeeccc19a504e7984b293127766daf0 P 5ce671f028a34ea4b8f0520c2689c9a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:20.517233 2668 raft_consensus.cc:2454] T 0a2f98dd905d40959dc676f94a4a595b P 5ce671f028a34ea4b8f0520c2689c9a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:20.537470 2805 tablet_service.cc:1467] Processing CreateTablet for tablet bce14d014c34483fb45861f545930b51 (DEFAULT_TABLE table=default.Table_d1cbbe2de1e64a64bda8507e7ac034bc [id=45bfe0ebb69343e89348dc2eba563c85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.538416 2805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bce14d014c34483fb45861f545930b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.538625 2937 tablet_service.cc:1467] Processing CreateTablet for tablet bce14d014c34483fb45861f545930b51 (DEFAULT_TABLE table=default.Table_d1cbbe2de1e64a64bda8507e7ac034bc [id=45bfe0ebb69343e89348dc2eba563c85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.533736 4107 raft_consensus.cc:504] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:20.544938 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 116956us
I20260927 23:28:20.550027 2943 tablet_service.cc:1467] Processing CreateTablet for tablet b86d8426e4494d39ab16228b8cfcfad4 (DEFAULT_TABLE table=default.Table_1e6962ace257421683c16954e0f6b0b8 [id=93f1bf71ba5d4d7093a0ecd3a9f0bc1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.555640 4112 consensus_queue.cc:1035] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:20.556356 2794 tablet_service.cc:1467] Processing CreateTablet for tablet b86d8426e4494d39ab16228b8cfcfad4 (DEFAULT_TABLE table=default.Table_1e6962ace257421683c16954e0f6b0b8 [id=93f1bf71ba5d4d7093a0ecd3a9f0bc1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.559960 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b86d8426e4494d39ab16228b8cfcfad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.569350 2807 tablet_service.cc:1467] Processing CreateTablet for tablet b72b96808c1c47909b468f94f68bfef1 (DEFAULT_TABLE table=default.Table_5d8d3564c4d844699dde7932dc4cb4e4 [id=cfe3ab6933794a50844075be6951c092]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.570760 2964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31676cfbbad04e728adb4b8aa8d905a6" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
W20260927 23:28:20.571987 2761 leader_election.cc:341] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:20.572448 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b72b96808c1c47909b468f94f68bfef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.577198 4068 raft_consensus.cc:504] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:20.578763 2609 leader_election.cc:341] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:20.579169 2609 leader_election.cc:302] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ce671f028a34ea4b8f0520c2689c9a1; no voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d
I20260927 23:28:20.546280 2668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31676cfbbad04e728adb4b8aa8d905a6" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:20.580375 2668 raft_consensus.cc:3037] T 31676cfbbad04e728adb4b8aa8d905a6 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:20.584658 4107 leader_election.cc:288] T 31676cfbbad04e728adb4b8aa8d905a6 P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:20.581689 2645 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d5b59f73974430c91ba19c2ec307ea0 with delete_type TABLET_DATA_DELETED (replaced by tablet d8a335cd96dc4ae4af4207049495327f) from {username='slave'} at 127.0.0.1:54604
I20260927 23:28:20.582928 2646 tablet_service.cc:1467] Processing CreateTablet for tablet 24900c5a9d154a90ac592034a86d0a37 (DEFAULT_TABLE table=default.Table_4ea12f0a3fb449528dc3a1a406b707b6 [id=7858e7c0328b4fd5ae53ce8427ca5305]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.593653 2646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24900c5a9d154a90ac592034a86d0a37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.584254 2648 tablet_service.cc:1467] Processing CreateTablet for tablet 8e0e136fd83240369a5173b6b077ab9f (DEFAULT_TABLE table=default.Table_f5c7bbc6853d4806aca1ad9a751e1e4b [id=6b7de9bcff23494eba44920db81a9f4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.597011 2648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e0e136fd83240369a5173b6b077ab9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.586206 2650 tablet_service.cc:1467] Processing CreateTablet for tablet 2e045145d4ee4654b987f6101bd178ad (DEFAULT_TABLE table=default.Table_b7548cd9beb941bd83aa4939fa976df2 [id=7216d324b57e4efca64d3e60c6318660]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.601030 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e045145d4ee4654b987f6101bd178ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.588351 2644 tablet_service.cc:1467] Processing CreateTablet for tablet 561e72d79f1e405c993d0145a892ce72 (DEFAULT_TABLE table=default.Table_d81346de05884027980a011d83c14bd2 [id=2988c6c13fc240c3973738ad329ad5a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.605034 2644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 561e72d79f1e405c993d0145a892ce72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.590159 2652 tablet_service.cc:1467] Processing CreateTablet for tablet b72b96808c1c47909b468f94f68bfef1 (DEFAULT_TABLE table=default.Table_5d8d3564c4d844699dde7932dc4cb4e4 [id=cfe3ab6933794a50844075be6951c092]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.609036 2652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b72b96808c1c47909b468f94f68bfef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.611574 3167 tablet_bootstrap.cc:492] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:20.612510 4160 tablet.cc:1722] T 6ca42b47d8044adc99a82859bf79e606 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 3
I20260927 23:28:20.639518 4123 consensus_queue.cc:1035] T f253a54ab3b44822aee266314fa45495 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:20.645290 4068 leader_election.cc:288] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Requested vote from peers abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573), 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657)
I20260927 23:28:20.651470 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 40bce5362d2541329b253716bbd99e07 (table default.Table_82bad979f0da4702af4e135570772d7e [id=9dec5171c0de4e9cb35f2c21342f4e6c]). Expected version 1 got 0
I20260927 23:28:20.661123 3072 ts_tablet_manager.cc:1428] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.265s user 0.004s sys 0.002s
W20260927 23:28:20.664613 2608 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:33209, user_credentials={real_user=slave}} blocked reactor thread for 95963.8us
I20260927 23:28:20.669194 4001 raft_consensus.cc:2726] T ee5d073c8df84f0884dc0f0d88eeed9f P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:20.669974 2609 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:41573, user_credentials={real_user=slave}} blocked reactor thread for 91236.7us
I20260927 23:28:20.670874 2934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e0e136fd83240369a5173b6b077ab9f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:20.663959 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 213396us
I20260927 23:28:20.676766 2932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e045145d4ee4654b987f6101bd178ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.681306 2938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 561e72d79f1e405c993d0145a892ce72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.681414 2670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119a1338b04740d0bbf291c8fbe34992" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" is_pre_election: true
I20260927 23:28:20.681970 2670 raft_consensus.cc:2454] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 0.
I20260927 23:28:20.685024 2933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b72b96808c1c47909b468f94f68bfef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.685011 4151 tablet.cc:1722] T 1c87484b8b9742faa683f13d37308e58 P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:20.687201 2937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bce14d014c34483fb45861f545930b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.689024 4163 tablet.cc:1722] T 6d0d8a1f277242588f57e4df95df5d5a P 5ce671f028a34ea4b8f0520c2689c9a1: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:20.692693 4105 tablet.cc:1722] T 1c87484b8b9742faa683f13d37308e58 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:20.694345 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 9eb27269a590476f878ee52a2aefd141 (table default.Table_9f833d9437504850b434e8c3ceb6662a [id=8beb1e4757014f69ba48b5ea31b27afc]). Expected version 1 got 0
I20260927 23:28:20.694321 2649 tablet_service.cc:1467] Processing CreateTablet for tablet bce14d014c34483fb45861f545930b51 (DEFAULT_TABLE table=default.Table_d1cbbe2de1e64a64bda8507e7ac034bc [id=45bfe0ebb69343e89348dc2eba563c85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.695102 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 1c87484b8b9742faa683f13d37308e58 (table default.Table_4fa6aec871b84978973cb579e195404b [id=2cd91cbdb5b14e16b847253971c8cb25]). Expected version 1 got 0
I20260927 23:28:20.695254 2649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bce14d014c34483fb45861f545930b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.695727 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 5c87d497da884544920b3be9c78daccf (table default.Table_e007bebc730046b48b94b18f382b2cb7 [id=9aaee6c6f55b45aeab373b0a68a5c6cd]). Expected version 1 got 0
W20260927 23:28:20.696465 2378 catalog_manager.cc:4821] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet 4d5b59f73974430c91ba19c2ec307ea0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 4d5b59f73974430c91ba19c2ec307ea0 already in progress: creating tablet
I20260927 23:28:20.700670 2820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119a1338b04740d0bbf291c8fbe34992" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abe182b2afc64d908f692ef45eae1d51"
I20260927 23:28:20.701207 2820 raft_consensus.cc:3037] T 119a1338b04740d0bbf291c8fbe34992 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:20.705387 2645 tablet_service.cc:1467] Processing CreateTablet for tablet b86d8426e4494d39ab16228b8cfcfad4 (DEFAULT_TABLE table=default.Table_1e6962ace257421683c16954e0f6b0b8 [id=93f1bf71ba5d4d7093a0ecd3a9f0bc1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:20.706732 2645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b86d8426e4494d39ab16228b8cfcfad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.706986 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 2d0d8a82c5234880ab8396b1777e36c8 (table default.Table_ce63ee103a6c4ae49b4f49360d1e1195 [id=c7a02939c757489cb0b7318f5ed346ec]). Expected version 2 got 0
I20260927 23:28:20.708003 2388 catalog_manager.cc:5587] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657) does not have the latest schema for tablet 49883297d2b046b99f2ec1343c276ff9 (table default.Table_4458b554cada4d42bc1307905a783181 [id=adbf504950b64a08996f102f4ac2d662]). Expected version 1 got 0
I20260927 23:28:20.692763 2943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b86d8426e4494d39ab16228b8cfcfad4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:20.723541 3070 tablet_bootstrap.cc:492] T f58349edf0db40f98951e116fc2582b0 P 5ce671f028a34ea4b8f0520c2689c9a1: No bootstrap required, opened a new log
I20260927 23:28:20.723969 3070 ts_tablet_manager.cc:1397] T f58349edf0db40f98951e116fc2582b0 P 5ce671f028a34ea4b8f0520c2689c9a1: Time spent bootstrapping tablet: real 0.410s user 0.014s sys 0.001s
I20260927 23:28:20.725369 2380 catalog_manager.cc:4841] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): tablet 9670df27c4404acda97661c69420b5cf (table default.Table_fe4babd936e548c7906748e495d790a1 [id=1786524f92bb47f4b74e15df9bc77e98]) successfully deleted
W20260927 23:28:20.728078 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 49949us
I20260927 23:28:20.732528 2389 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_a5f6638fb0a940fcb37e20213e3d081f [id=47c1f33b77964104b1be5f25b788d8af] alter complete (version 2)
I20260927 23:28:20.733151 2387 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: Sending DeleteTablet for 3 replicas of tablet 6d0d8a1f277242588f57e4df95df5d5a
I20260927 23:28:20.733402 2391 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f: default.Table_8eff0fa7f4994841bef399e5f67f7dca [id=687de3cd507b4b5e82ba6df82a290757] alter complete (version 2)
I20260927 23:28:20.734490 2387 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:51972 (request call id 325) took 1085 ms. Trace:
I20260927 23:28:20.734757 2387 rpcz_store.cc:276] 0927 23:28:19.649131 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:19.679778 (+ 30647us) service_pool.cc:227] Handling call
0927 23:28:19.750135 (+ 70357us) spinlock_profiling.cc:243] Waited 69 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b3466 00007f8bb3bf4b34 00007f8bb3bf4ac2 00007f8bb3bf49f1 00007f8bb3bf496c 00007f8bb3bf3c22 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:19.751628 (+ 1493us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:28:19.752612 (+ 984us) catalog_manager.cc:2752] Modifying in-memory table state
0927 23:28:19.752882 (+ 270us) catalog_manager.cc:2761] Locking tablets
0927 23:28:19.956984 (+204102us) catalog_manager.cc:2775] Removing table and tablets from system table
0927 23:28:19.998327 (+ 41343us) catalog_manager.cc:2793] Removing table from by-name map
0927 23:28:19.998580 (+ 253us) catalog_manager.cc:2809] Committing in-memory state
0927 23:28:20.733063 (+734483us) catalog_manager.cc:2813] Aborting table tasks
0927 23:28:20.734439 (+ 1376us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:19.961092 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:19.961485 (+ 393us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:19.961514 (+ 29us) write_op.cc:435] Acquired schema lock
0927 23:28:19.961528 (+ 14us) tablet.cc:660] Decoding operations
0927 23:28:19.961854 (+ 326us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:19.961908 (+ 54us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:19.961931 (+ 23us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:19.962217 (+ 286us) tablet.cc:699] Row locks acquired
0927 23:28:19.962233 (+ 16us) write_op.cc:260] PREPARE: finished
0927 23:28:19.962350 (+ 117us) write_op.cc:270] Start()
0927 23:28:19.962436 (+ 86us) write_op.cc:276] Timestamp: P: 1790551699962292 usec, L: 0
0927 23:28:19.962455 (+ 19us) op_driver.cc:331] REPLICATION: starting
0927 23:28:19.962968 (+ 513us) log.cc:842] Serialized 787 byte log entry
0927 23:28:19.995874 (+ 32906us) op_driver.cc:444] REPLICATION: finished
0927 23:28:19.996177 (+ 303us) write_op.cc:301] APPLY: starting
0927 23:28:19.996228 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:19.996348 (+ 120us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:19.996461 (+ 113us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:19.996700 (+ 239us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:19.996839 (+ 139us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:19.996883 (+ 44us) write_op.cc:312] APPLY: finished
0927 23:28:19.997060 (+ 177us) log.cc:842] Serialized 29 byte log entry
0927 23:28:19.997209 (+ 149us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:19.997282 (+ 73us) write_op.cc:454] Released schema lock
0927 23:28:19.997313 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:19.997488 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:19.997512 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"spinlock_wait_cycles":158625664,"child_traces":[["op",{"apply.queue_time_us":174,"apply.run_cpu_time_us":1511,"apply.run_wall_time_us":1511,"num_ops":2,"prepare.queue_time_us":2340,"prepare.run_cpu_time_us":2085,"prepare.run_wall_time_us":2085,"replication_time_us":33348}]]}
I20260927 23:28:20.740064 2674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "119a1338b04740d0bbf291c8fbe34992" candidate_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ce671f028a34ea4b8f0520c2689c9a1"
I20260927 23:28:20.740504 2805 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d0d8a1f277242588f57e4df95df5d5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:19 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:20.740648 2674 raft_consensus.cc:3037] T 119a1338b04740d0bbf291c8fbe34992 P 5ce671f028a34ea4b8f0520c2689c9a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:20.741434 2276 master-stress-test.cc:580] Killed test thread 7/10
I20260927 23:28:20.741670 2276 master-stress-test.cc:578] Killing test thread 8/10
I20260927 23:28:20.741954 2276 master-stress-test.cc:580] Killed test thread 8/10
I20260927 23:28:20.742131 2276 master-stress-test.cc:578] Killing test thread 9/10
I20260927 23:28:20.742401 2276 master-stress-test.cc:580] Killed test thread 9/10
I20260927 23:28:20.742573 2276 master-stress-test.cc:578] Killing test thread 10/10
I20260927 23:28:20.742835 2276 master-stress-test.cc:580] Killed test thread 10/10
I20260927 23:28:20.743130 2276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu with pid 2554
I20260927 23:28:20.743924 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d0d8a1f277242588f57e4df95df5d5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:19 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:20.746179 4150 tablet_replica.cc:330] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:20.746965 4150 raft_consensus.cc:2229] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:20.747427 4150 raft_consensus.cc:2258] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:20.748829 2947 tablet_service.cc:1514] Processing DeleteTablet for tablet f58349edf0db40f98951e116fc2582b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:44952
I20260927 23:28:20.750804 4150 ts_tablet_manager.cc:1905] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:20.757927 2384 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 35) took 794 ms (client timeout 1000 ms). Trace:
W20260927 23:28:20.758338 2384 rpcz_store.cc:269] 0927 23:28:19.963938 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:19.964059 (+ 121us) service_pool.cc:227] Handling call
0927 23:28:20.757865 (+793806us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":17662,"reported_tablets":13}
I20260927 23:28:20.759949 2820 raft_consensus.cc:2454] T 119a1338b04740d0bbf291c8fbe34992 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de216a4a8e4b4e1b857d8647a3b05f6d in term 1.
I20260927 23:28:20.766391 4123 raft_consensus.cc:482] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:20.766947 4123 raft_consensus.cc:504] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.770131 2804 tablet_service.cc:1514] Processing DeleteTablet for tablet bbeeccc19a504e7984b293127766daf0 with delete_type TABLET_DATA_DELETED (replaced by tablet e268bcb4b0614f37a4d9d96f8f40d1cb) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:20.770707 2798 tablet_service.cc:1514] Processing DeleteTablet for tablet a1890852164d492db3540a97847888fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:20.769606 2906 leader_election.cc:302] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
W20260927 23:28:20.772146 2389 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 51) took 3006 ms (client timeout 1000 ms). Trace:
I20260927 23:28:20.775658 4123 leader_election.cc:288] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
W20260927 23:28:20.777799 2386 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:33427 (request call id 34) took 1969 ms (client timeout 1000 ms). Trace:
W20260927 23:28:20.778177 2386 rpcz_store.cc:269] 0927 23:28:18.808543 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:18.809333 (+ 790us) service_pool.cc:227] Handling call
0927 23:28:20.777737 (+1968404us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":42170,"reported_tablets":10}
I20260927 23:28:20.788199 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cd0972f635d4979abf5fd30e9b02b1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:44952
W20260927 23:28:20.799783 2392 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 52) took 1831 ms (client timeout 1000 ms). Trace:
I20260927 23:28:20.785071 4161 raft_consensus.cc:2781] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:20.800675 4161 raft_consensus.cc:686] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Becoming Leader. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Running, Role: LEADER
I20260927 23:28:20.801381 4161 consensus_queue.cc:237] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:20.772555 2389 rpcz_store.cc:269] 0927 23:28:17.761280 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:17.904143 (+142863us) service_pool.cc:227] Handling call
0927 23:28:20.767409 (+2863266us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:18.437759 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:18.438215 (+ 456us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:18.438245 (+ 30us) write_op.cc:435] Acquired schema lock
0927 23:28:18.438256 (+ 11us) tablet.cc:660] Decoding operations
0927 23:28:18.438564 (+ 308us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:18.438612 (+ 48us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:18.438635 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:18.438800 (+ 165us) tablet.cc:699] Row locks acquired
0927 23:28:18.438813 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:28:18.438945 (+ 132us) write_op.cc:270] Start()
0927 23:28:18.439038 (+ 93us) write_op.cc:276] Timestamp: P: 1790551698438869 usec, L: 0
0927 23:28:18.439053 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:18.439931 (+ 878us) log.cc:842] Serialized 524 byte log entry
0927 23:28:18.497769 (+ 57838us) op_driver.cc:444] REPLICATION: finished
0927 23:28:18.519176 (+ 21407us) write_op.cc:301] APPLY: starting
0927 23:28:18.519250 (+ 74us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:18.519352 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:18.519464 (+ 112us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:18.519688 (+ 224us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:18.519848 (+ 160us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:18.519893 (+ 45us) write_op.cc:312] APPLY: finished
0927 23:28:18.520169 (+ 276us) log.cc:842] Serialized 23 byte log entry
0927 23:28:18.520353 (+ 184us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:18.520427 (+ 74us) write_op.cc:454] Released schema lock
0927 23:28:18.520460 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:18.520642 (+ 182us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:18.520662 (+ 20us) write_op.cc:454] Released schema lock
Related trace 'op':
0927 23:28:18.755180 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:18.755557 (+ 377us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:18.755580 (+ 23us) write_op.cc:435] Acquired schema lock
0927 23:28:18.755595 (+ 15us) tablet.cc:660] Decoding operations
0927 23:28:18.755888 (+ 293us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:18.755921 (+ 33us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:18.755951 (+ 30us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:18.756094 (+ 143us) tablet.cc:699] Row locks acquired
0927 23:28:18.756112 (+ 18us) write_op.cc:260] PREPARE: finished
0927 23:28:18.756220 (+ 108us) write_op.cc:270] Start()
0927 23:28:18.764352 (+ 8132us) write_op.cc:276] Timestamp: P: 1790551698756177 usec, L: 0
0927 23:28:18.764384 (+ 32us) op_driver.cc:331] REPLICATION: starting
0927 23:28:18.764985 (+ 601us) log.cc:842] Serialized 356 byte log entry
0927 23:28:19.085617 (+320632us) op_driver.cc:444] REPLICATION: finished
0927 23:28:19.086409 (+ 792us) write_op.cc:301] APPLY: starting
0927 23:28:19.086482 (+ 73us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:19.086586 (+ 104us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:19.086687 (+ 101us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:19.086887 (+ 200us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:19.087030 (+ 143us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:19.087075 (+ 45us) write_op.cc:312] APPLY: finished
0927 23:28:19.087291 (+ 216us) log.cc:842] Serialized 23 byte log entry
0927 23:28:19.087474 (+ 183us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:19.087541 (+ 67us) write_op.cc:454] Released schema lock
0927 23:28:19.087572 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:19.087889 (+ 317us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:19.087919 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20082,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":21247,"apply.run_cpu_time_us":1595,"apply.run_wall_time_us":1640,"num_ops":1,"prepare.queue_time_us":757,"prepare.run_cpu_time_us":2125,"prepare.run_wall_time_us":2405,"replication_time_us":58610,"thread_start_us":252656,"threads_started":1}],["op",{"apply.queue_time_us":604,"apply.run_cpu_time_us":1540,"apply.run_wall_time_us":1672,"num_ops":1,"prepare.queue_time_us":114824,"prepare.run_cpu_time_us":2000,"prepare.run_wall_time_us":10015,"replication_time_us":321123}]]}
I20260927 23:28:20.805346 3167 tablet_bootstrap.cc:700] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51: Time spent opening tablet: real 0.172s user 0.001s sys 0.000s
I20260927 23:28:20.806125 3167 tablet_bootstrap.cc:654] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:20.806417 2385 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:55769 (request call id 53) took 826 ms (client timeout 1000 ms). Trace:
I20260927 23:28:20.811456 2963 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38be781ead0e470e9928881594b4b98f" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
W20260927 23:28:20.813624 2761 leader_election.cc:341] T 38be781ead0e470e9928881594b4b98f P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:20.814016 2794 tablet_service.cc:1514] Processing DeleteTablet for tablet f58349edf0db40f98951e116fc2582b0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:20.814754 2795 tablet_service.cc:1514] Processing DeleteTablet for tablet 8cd0972f635d4979abf5fd30e9b02b1d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:08 UTC) from {username='slave'} at 127.0.0.1:42492
I20260927 23:28:20.818162 4123 consensus_queue.cc:1035] T 50dd9944d3134ab9b23a6109fd7d6ee5 P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:20.820865 4137 tablet.cc:1722] T 49883297d2b046b99f2ec1343c276ff9 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:20.822382 3072 tablet_bootstrap.cc:492] T 38be781ead0e470e9928881594b4b98f P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:20.828121 4150 ts_tablet_manager.cc:1918] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:20.829399 4150 log.cc:1196] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/6d0d8a1f277242588f57e4df95df5d5a
I20260927 23:28:20.830776 4150 ts_tablet_manager.cc:1939] T 6d0d8a1f277242588f57e4df95df5d5a P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
W20260927 23:28:20.800388 2392 rpcz_store.cc:269] 0927 23:28:18.968146 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:19.061590 (+ 93444us) service_pool.cc:227] Handling call
0927 23:28:19.119574 (+ 57984us) spinlock_profiling.cc:243] Waited 52.4 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:20.799718 (+1680144us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:19.829880 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:19.830302 (+ 422us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:19.830328 (+ 26us) write_op.cc:435] Acquired schema lock
0927 23:28:19.830343 (+ 15us) tablet.cc:660] Decoding operations
0927 23:28:19.830654 (+ 311us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:19.830691 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:19.830720 (+ 29us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:19.911846 (+ 81126us) tablet.cc:699] Row locks acquired
0927 23:28:19.911869 (+ 23us) write_op.cc:260] PREPARE: finished
0927 23:28:19.912016 (+ 147us) write_op.cc:270] Start()
0927 23:28:19.912367 (+ 351us) write_op.cc:276] Timestamp: P: 1790551699911960 usec, L: 0
0927 23:28:19.912391 (+ 24us) op_driver.cc:331] REPLICATION: starting
0927 23:28:19.913156 (+ 765us) log.cc:842] Serialized 524 byte log entry
0927 23:28:19.945783 (+ 32627us) op_driver.cc:444] REPLICATION: finished
0927 23:28:19.946065 (+ 282us) write_op.cc:301] APPLY: starting
0927 23:28:19.946117 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:19.946324 (+ 207us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:19.946424 (+ 100us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:19.946621 (+ 197us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:19.946814 (+ 193us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:19.946855 (+ 41us) write_op.cc:312] APPLY: finished
0927 23:28:19.947036 (+ 181us) log.cc:842] Serialized 23 byte log entry
0927 23:28:19.947653 (+ 617us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:19.947711 (+ 58us) write_op.cc:454] Released schema lock
0927 23:28:19.947744 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:19.948857 (+ 1113us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:19.948881 (+ 24us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15424,"reported_tablets":24,"spinlock_wait_cycles":120621312,"child_traces":[["op",{"apply.queue_time_us":149,"apply.run_cpu_time_us":1409,"apply.run_wall_time_us":2969,"num_ops":1,"prepare.queue_time_us":47207,"prepare.run_cpu_time_us":2298,"prepare.run_wall_time_us":99171,"replication_time_us":33304}]]}
I20260927 23:28:20.833427 4105 tablet.cc:1722] T 49883297d2b046b99f2ec1343c276ff9 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:28:20.841336 2946 tablet_service.cc:1514] Processing DeleteTablet for tablet a1890852164d492db3540a97847888fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:28:09 UTC) from {username='slave'} at 127.0.0.1:44952
W20260927 23:28:20.806789 2385 rpcz_store.cc:269] 0927 23:28:19.979542 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:19.980264 (+ 722us) service_pool.cc:227] Handling call
0927 23:28:20.806354 (+826090us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:20.540466 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:20.540801 (+ 335us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:20.540827 (+ 26us) write_op.cc:435] Acquired schema lock
0927 23:28:20.540839 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:20.541063 (+ 224us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:20.541090 (+ 27us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:20.541102 (+ 12us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:20.541281 (+ 179us) tablet.cc:699] Row locks acquired
0927 23:28:20.541290 (+ 9us) write_op.cc:260] PREPARE: finished
0927 23:28:20.541366 (+ 76us) write_op.cc:270] Start()
0927 23:28:20.541440 (+ 74us) write_op.cc:276] Timestamp: P: 1790551700541299 usec, L: 0
0927 23:28:20.541455 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:20.541918 (+ 463us) log.cc:842] Serialized 895 byte log entry
0927 23:28:20.552832 (+ 10914us) op_driver.cc:444] REPLICATION: finished
0927 23:28:20.553125 (+ 293us) write_op.cc:301] APPLY: starting
0927 23:28:20.553174 (+ 49us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:20.553277 (+ 103us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:20.553360 (+ 83us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:20.553598 (+ 238us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:20.553723 (+ 125us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0927 23:28:20.553765 (+ 42us) write_op.cc:312] APPLY: finished
0927 23:28:20.553934 (+ 169us) log.cc:842] Serialized 29 byte log entry
0927 23:28:20.554077 (+ 143us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:20.554143 (+ 66us) write_op.cc:454] Released schema lock
0927 23:28:20.554171 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:20.554325 (+ 154us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:20.554347 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":47399,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":167,"apply.run_cpu_time_us":1390,"apply.run_wall_time_us":1390,"num_ops":2,"prepare.queue_time_us":1625,"prepare.run_cpu_time_us":1710,"prepare.run_wall_time_us":1710,"raft.queue_time_us":155,"raft.run_cpu_time_us":587,"raft.run_wall_time_us":587,"replication_time_us":11292}]]}
I20260927 23:28:20.849329 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 6d0d8a1f277242588f57e4df95df5d5a (table default.Table_47fc9fc598ce4ca4bb4f94a523b39f74 [id=7b1bf5063eef401eb9de9b6f7aa40e4b]) successfully deleted
I20260927 23:28:20.861047 4150 tablet_replica.cc:330] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:20.861897 4150 raft_consensus.cc:2229] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:20.862800 4150 pending_rounds.cc:62] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d: Trying to abort 1 pending ops.
I20260927 23:28:20.863022 4150 pending_rounds.cc:68] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099762476302336 op_type: NO_OP noop_request { }
I20260927 23:28:20.863390 4150 raft_consensus.cc:2866] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:28:20.863730 4150 raft_consensus.cc:2258] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:20.888353 4105 tablet.cc:1722] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:20.901513 4137 tablet.cc:1722] T 2d0d8a82c5234880ab8396b1777e36c8 P abe182b2afc64d908f692ef45eae1d51: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20260927 23:28:20.916620 3072 tablet_bootstrap.cc:654] T 38be781ead0e470e9928881594b4b98f P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:20.925522 3167 tablet_bootstrap.cc:492] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:20.925917 3167 ts_tablet_manager.cc:1397] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.315s user 0.006s sys 0.003s
I20260927 23:28:20.927980 3167 raft_consensus.cc:348] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.928668 3167 raft_consensus.cc:374] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:20.928908 3167 raft_consensus.cc:729] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:20.929093 4148 tablet_replica.cc:330] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:20.929961 4148 raft_consensus.cc:2229] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:20.930786 4148 raft_consensus.cc:2258] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:20.933390 4148 ts_tablet_manager.cc:1905] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:20.933691 3167 consensus_queue.cc:260] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.936283 3167 ts_tablet_manager.cc:1428] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.010s user 0.005s sys 0.001s
I20260927 23:28:20.937143 3167 tablet_bootstrap.cc:492] T a8df6d0652634def9c8946c346dea702 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:20.899648 4150 ts_tablet_manager.cc:1905] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:20.949738 3167 tablet_bootstrap.cc:654] T a8df6d0652634def9c8946c346dea702 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:20.953991 3072 tablet_bootstrap.cc:492] T 38be781ead0e470e9928881594b4b98f P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:20.954396 3072 ts_tablet_manager.cc:1397] T 38be781ead0e470e9928881594b4b98f P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.134s user 0.008s sys 0.000s
I20260927 23:28:20.965657 3167 tablet_bootstrap.cc:492] T a8df6d0652634def9c8946c346dea702 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:20.966131 3167 ts_tablet_manager.cc:1397] T a8df6d0652634def9c8946c346dea702 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.029s user 0.010s sys 0.002s
I20260927 23:28:20.974758 4125 raft_consensus.cc:482] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:20.981067 4125 raft_consensus.cc:504] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.982885 4125 leader_election.cc:288] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:20.974473 3167 raft_consensus.cc:348] T a8df6d0652634def9c8946c346dea702 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.985540 3167 raft_consensus.cc:374] T a8df6d0652634def9c8946c346dea702 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:20.986114 3167 raft_consensus.cc:729] T a8df6d0652634def9c8946c346dea702 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:20.987273 3167 consensus_queue.cc:260] T a8df6d0652634def9c8946c346dea702 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.989311 2962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee5d073c8df84f0884dc0f0d88eeed9f" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" is_pre_election: true
I20260927 23:28:20.989961 2962 raft_consensus.cc:2454] T ee5d073c8df84f0884dc0f0d88eeed9f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 0.
I20260927 23:28:20.990749 3167 ts_tablet_manager.cc:1428] T a8df6d0652634def9c8946c346dea702 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.024s user 0.004s sys 0.000s
I20260927 23:28:20.992614 4176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:28:20.992564 3072 raft_consensus.cc:348] T 38be781ead0e470e9928881594b4b98f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:20.991667 2761 leader_election.cc:302] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:20.993207 3072 raft_consensus.cc:374] T 38be781ead0e470e9928881594b4b98f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:20.993428 3072 raft_consensus.cc:729] T 38be781ead0e470e9928881594b4b98f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:20.993541 4125 raft_consensus.cc:2781] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:20.993856 4125 raft_consensus.cc:482] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:20.994138 4125 raft_consensus.cc:3037] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:20.994053 3072 consensus_queue.cc:260] T 38be781ead0e470e9928881594b4b98f P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:20.995998 2391 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:39229 (request call id 38) took 2119 ms (client timeout 1000 ms). Trace:
I20260927 23:28:20.996338 3072 ts_tablet_manager.cc:1428] T 38be781ead0e470e9928881594b4b98f P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.042s user 0.004s sys 0.003s
I20260927 23:28:21.011231 4176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:28:21.004272 4148 ts_tablet_manager.cc:1918] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:28:21.012270 4148 log.cc:1196] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/6d0d8a1f277242588f57e4df95df5d5a
I20260927 23:28:21.011020 3167 tablet_bootstrap.cc:492] T 1542b6be05634f6a88694df446373fe9 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:21.010317 4125 raft_consensus.cc:504] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.015026 4125 leader_election.cc:288] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Requested vote from peers 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657), de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209)
I20260927 23:28:21.015594 4148 ts_tablet_manager.cc:1939] T 6d0d8a1f277242588f57e4df95df5d5a P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:21.015631 2962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee5d073c8df84f0884dc0f0d88eeed9f" candidate_uuid: "abe182b2afc64d908f692ef45eae1d51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d"
I20260927 23:28:21.016165 2962 raft_consensus.cc:3037] T ee5d073c8df84f0884dc0f0d88eeed9f P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:21.020232 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet 6d0d8a1f277242588f57e4df95df5d5a (table default.Table_47fc9fc598ce4ca4bb4f94a523b39f74 [id=7b1bf5063eef401eb9de9b6f7aa40e4b]) successfully deleted
I20260927 23:28:21.021965 2326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 315 } } ignore_live_leader: false dest_uuid: "bee21c28e51643189178a3c851672546" is_pre_election: true
I20260927 23:28:21.023599 2962 raft_consensus.cc:2454] T ee5d073c8df84f0884dc0f0d88eeed9f P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abe182b2afc64d908f692ef45eae1d51 in term 1.
I20260927 23:28:21.024622 2761 leader_election.cc:302] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abe182b2afc64d908f692ef45eae1d51, de216a4a8e4b4e1b857d8647a3b05f6d; no voters:
I20260927 23:28:21.025000 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" candidate_term: 4 candidate_status { last_received { term: 3 index: 315 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:28:21.025584 4125 raft_consensus.cc:2781] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:21.026031 4125 raft_consensus.cc:686] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Becoming Leader. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Running, Role: LEADER
I20260927 23:28:21.026477 3695 leader_election.cc:302] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150; no voters: 19a88ff0709d4266b933cc526577127f, bee21c28e51643189178a3c851672546
I20260927 23:28:21.026829 4125 consensus_queue.cc:237] T ee5d073c8df84f0884dc0f0d88eeed9f P abe182b2afc64d908f692ef45eae1d51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.028416 4148 tablet_replica.cc:330] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:21.029258 4148 raft_consensus.cc:2229] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:21.030006 4148 raft_consensus.cc:2258] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:21.032684 4176 leader_election.cc:288] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers bee21c28e51643189178a3c851672546 (127.2.57.62:38773), 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987)
I20260927 23:28:21.036773 4148 ts_tablet_manager.cc:1905] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:21.040858 3072 tablet_bootstrap.cc:492] T 31676cfbbad04e728adb4b8aa8d905a6 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:21.033942 4176 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:28:21.049670 3072 tablet_bootstrap.cc:654] T 31676cfbbad04e728adb4b8aa8d905a6 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:21.051461 3167 tablet_bootstrap.cc:654] T 1542b6be05634f6a88694df446373fe9 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:20.996313 2391 rpcz_store.cc:269] 0927 23:28:18.876603 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:18.969811 (+ 93208us) service_pool.cc:227] Handling call
0927 23:28:19.160353 (+190542us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b54000118d4. stack: 00007f8bad15c033 00007f8bad15bf30 00007f8baccdd534 00000000004db2b4 00000000004db249 00007f8bb4068af0 00007f8bb40ebf2a 00007f8bb5fb7d6c 00007f8bb60b1c3e 00007f8bb3be8fb4 00007f8bb3be8f42 00007f8bb3be8e71 00007f8bb3be8dec 00007f8bb3be80a2 00007f8bb1e9294c 00007f8bb1e91d86
0927 23:28:20.118152 (+957799us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:20.118178 (+ 26us) write_op.cc:454] Released schema lock
0927 23:28:20.995932 (+877754us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:20.098660 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:20.099096 (+ 436us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:20.099124 (+ 28us) write_op.cc:435] Acquired schema lock
0927 23:28:20.099136 (+ 12us) tablet.cc:660] Decoding operations
0927 23:28:20.099409 (+ 273us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:20.099458 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:20.099483 (+ 25us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:28:20.099629 (+ 146us) tablet.cc:699] Row locks acquired
0927 23:28:20.099642 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:28:20.099762 (+ 120us) write_op.cc:270] Start()
0927 23:28:20.099852 (+ 90us) write_op.cc:276] Timestamp: P: 1790551700099690 usec, L: 0
0927 23:28:20.099867 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:28:20.100404 (+ 537us) log.cc:842] Serialized 356 byte log entry
0927 23:28:20.108389 (+ 7985us) op_driver.cc:444] REPLICATION: finished
0927 23:28:20.108686 (+ 297us) write_op.cc:301] APPLY: starting
0927 23:28:20.108737 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:20.108823 (+ 86us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:20.108920 (+ 97us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:20.109061 (+ 141us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:20.109177 (+ 116us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:28:20.109217 (+ 40us) write_op.cc:312] APPLY: finished
0927 23:28:20.109375 (+ 158us) log.cc:842] Serialized 23 byte log entry
0927 23:28:20.109507 (+ 132us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:20.109565 (+ 58us) write_op.cc:454] Released schema lock
0927 23:28:20.109592 (+ 27us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":44267,"reported_tablets":33,"spinlock_wait_cycles":112323456,"child_traces":[["op",{"apply.queue_time_us":173,"apply.run_cpu_time_us":1066,"apply.run_wall_time_us":1065,"num_ops":1,"prepare.queue_time_us":13102,"prepare.run_cpu_time_us":2034,"prepare.run_wall_time_us":2033,"raft.queue_time_us":162,"raft.run_cpu_time_us":627,"raft.run_wall_time_us":626,"replication_time_us":8446}]]}
I20260927 23:28:21.084419 4180 raft_consensus.cc:482] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:28:21.085155 4180 raft_consensus.cc:504] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:28:21.088433 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 4 candidate_status { last_received { term: 3 index: 390 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f" is_pre_election: true
I20260927 23:28:21.088804 3072 tablet_bootstrap.cc:492] T 31676cfbbad04e728adb4b8aa8d905a6 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:21.089193 3072 ts_tablet_manager.cc:1397] T 31676cfbbad04e728adb4b8aa8d905a6 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.049s user 0.009s sys 0.000s
I20260927 23:28:21.091445 3072 raft_consensus.cc:348] T 31676cfbbad04e728adb4b8aa8d905a6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.093415 3072 raft_consensus.cc:374] T 31676cfbbad04e728adb4b8aa8d905a6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:21.100397 3072 raft_consensus.cc:729] T 31676cfbbad04e728adb4b8aa8d905a6 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:21.101580 3167 tablet_bootstrap.cc:492] T 1542b6be05634f6a88694df446373fe9 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:21.101949 3167 ts_tablet_manager.cc:1397] T 1542b6be05634f6a88694df446373fe9 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.091s user 0.008s sys 0.000s
I20260927 23:28:21.104832 4180 leader_election.cc:288] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:28:21.101176 3072 consensus_queue.cc:260] T 31676cfbbad04e728adb4b8aa8d905a6 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:21.109243 2379 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 178503861b1648d9a931d1606d807ed8 (table default.Table_0df33121d2eb4445ac8b8c6a9ee44d24 [id=0a5810627a964e7caf66c69e7c3ec9b4], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 178503861b1648d9a931d1606d807ed8 consensus state has no leader
I20260927 23:28:21.103967 3167 raft_consensus.cc:348] T 1542b6be05634f6a88694df446373fe9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.111411 3167 raft_consensus.cc:374] T 1542b6be05634f6a88694df446373fe9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:21.111716 3167 raft_consensus.cc:729] T 1542b6be05634f6a88694df446373fe9 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:21.112648 3167 consensus_queue.cc:260] T 1542b6be05634f6a88694df446373fe9 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.115410 3072 ts_tablet_manager.cc:1428] T 31676cfbbad04e728adb4b8aa8d905a6 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.026s user 0.005s sys 0.000s
I20260927 23:28:21.116286 3072 tablet_bootstrap.cc:492] T db8996200d9c4c5f8572d1db2dbfe4e3 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:21.083427 4150 ts_tablet_manager.cc:1918] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:21.126837 4150 log.cc:1196] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/f58349edf0db40f98951e116fc2582b0
I20260927 23:28:21.127674 4150 ts_tablet_manager.cc:1939] T f58349edf0db40f98951e116fc2582b0 P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:21.128032 3167 ts_tablet_manager.cc:1428] T 1542b6be05634f6a88694df446373fe9 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.026s user 0.007s sys 0.000s
I20260927 23:28:21.128893 3167 tablet_bootstrap.cc:492] T 4d5b59f73974430c91ba19c2ec307ea0 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:21.134969 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet f58349edf0db40f98951e116fc2582b0 (table default.Table_0b205c7018ea4694b24bcfaeb93e859a [id=9878c4920bd7428d82faa9bb48c171a2]) successfully deleted
I20260927 23:28:21.145821 4150 tablet_replica.cc:330] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d: stopping tablet replica
I20260927 23:28:21.146880 4150 raft_consensus.cc:2229] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:28:21.147781 4150 raft_consensus.cc:2258] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:21.158948 4148 ts_tablet_manager.cc:1918] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:21.160746 4148 log.cc:1196] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/bbeeccc19a504e7984b293127766daf0
I20260927 23:28:21.164796 3167 tablet_bootstrap.cc:654] T 4d5b59f73974430c91ba19c2ec307ea0 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:21.161561 4148 ts_tablet_manager.cc:1939] T bbeeccc19a504e7984b293127766daf0 P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:21.174911 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet bbeeccc19a504e7984b293127766daf0 (table default.Table_27f9b741e6184aa0bb0f95975292a4ae [id=925b33a7e7404cd1bb45e7e9d9facfed]) successfully deleted
I20260927 23:28:21.184284 4148 tablet_replica.cc:330] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
I20260927 23:28:21.185274 4148 raft_consensus.cc:2229] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:21.186578 4148 raft_consensus.cc:2258] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:21.188675 4148 ts_tablet_manager.cc:1905] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:21.152410 3072 tablet_bootstrap.cc:654] T db8996200d9c4c5f8572d1db2dbfe4e3 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:21.206370 4148 ts_tablet_manager.cc:1918] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:21.223135 4150 ts_tablet_manager.cc:1905] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:21.226627 3695 connection.cc:535] server connection from 127.2.57.1:40507 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 23:28:21.233274 4148 log.cc:1196] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/a1890852164d492db3540a97847888fd
I20260927 23:28:21.235359 4148 ts_tablet_manager.cc:1939] T a1890852164d492db3540a97847888fd P abe182b2afc64d908f692ef45eae1d51: Deleting consensus metadata
I20260927 23:28:21.241923 3072 tablet_bootstrap.cc:492] T db8996200d9c4c5f8572d1db2dbfe4e3 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:21.242408 3072 ts_tablet_manager.cc:1397] T db8996200d9c4c5f8572d1db2dbfe4e3 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.126s user 0.009s sys 0.009s
I20260927 23:28:21.244570 3072 raft_consensus.cc:348] T db8996200d9c4c5f8572d1db2dbfe4e3 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.237739 3167 tablet_bootstrap.cc:492] T 4d5b59f73974430c91ba19c2ec307ea0 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
W20260927 23:28:21.244602 2761 connection.cc:535] server connection from 127.2.57.1:35131 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:28:21.245052 2906 connection.cc:535] server connection from 127.2.57.1:44363 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 23:28:21.245769 3167 ts_tablet_manager.cc:1397] T 4d5b59f73974430c91ba19c2ec307ea0 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.117s user 0.019s sys 0.004s
I20260927 23:28:21.246074 3072 raft_consensus.cc:374] T db8996200d9c4c5f8572d1db2dbfe4e3 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:21.246354 3072 raft_consensus.cc:729] T db8996200d9c4c5f8572d1db2dbfe4e3 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:21.247005 3072 consensus_queue.cc:260] T db8996200d9c4c5f8572d1db2dbfe4e3 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.247841 3167 raft_consensus.cc:348] T 4d5b59f73974430c91ba19c2ec307ea0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.248459 3167 raft_consensus.cc:374] T 4d5b59f73974430c91ba19c2ec307ea0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:21.248687 3167 raft_consensus.cc:729] T 4d5b59f73974430c91ba19c2ec307ea0 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:21.249285 3167 consensus_queue.cc:260] T 4d5b59f73974430c91ba19c2ec307ea0 P abe182b2afc64d908f692ef45eae1d51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
W20260927 23:28:21.252559 2907 connection.cc:535] client connection to 127.2.57.1:35657 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:28:21.252938 2907 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 23:28:21.257825 3167 ts_tablet_manager.cc:1428] T 4d5b59f73974430c91ba19c2ec307ea0 P abe182b2afc64d908f692ef45eae1d51: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
W20260927 23:28:21.258869 2380 connection.cc:535] client connection to 127.2.57.1:35657 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:28:21.259224 2380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 1 similar messages]
I20260927 23:28:21.259545 2379 catalog_manager.cc:4841] TS abe182b2afc64d908f692ef45eae1d51 (127.2.57.2:41573): tablet a1890852164d492db3540a97847888fd (table default.Table_49c3f2d8a1b14c02878df5f1934bbaba [id=1b4c77aab36b477593fa78f19be095af]) successfully deleted
I20260927 23:28:21.261900 3820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 4 candidate_status { last_received { term: 3 index: 390 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" is_pre_election: true
I20260927 23:28:21.262734 3820 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bee21c28e51643189178a3c851672546 in term 3.
I20260927 23:28:21.264518 2303 leader_election.cc:302] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150, bee21c28e51643189178a3c851672546; no voters: 19a88ff0709d4266b933cc526577127f
I20260927 23:28:21.269407 4180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260927 23:28:21.269855 4180 raft_consensus.cc:482] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 19a88ff0709d4266b933cc526577127f)
I20260927 23:28:21.270167 4180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 3 FOLLOWER]: Advancing to term 4
I20260927 23:28:21.271512 3167 tablet_bootstrap.cc:492] T 972763afd57847269fafadbdd2a5f3d6 P abe182b2afc64d908f692ef45eae1d51: Bootstrap starting.
I20260927 23:28:21.276502 4148 tablet_replica.cc:330] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51: stopping tablet replica
W20260927 23:28:21.276748 2304 connection.cc:535] server connection from 127.2.57.1:36953 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:28:21.278270 2763 connection.cc:535] client connection to 127.2.57.1:35657 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:28:21.278694 2378 connection.cc:535] server connection from 127.2.57.1:39229 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:28:21.278728 2763 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 23:28:21.264174 3072 ts_tablet_manager.cc:1428] T db8996200d9c4c5f8572d1db2dbfe4e3 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.021s user 0.004s sys 0.000s
I20260927 23:28:21.286229 3072 tablet_bootstrap.cc:492] T a8df6d0652634def9c8946c346dea702 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:21.288805 3167 tablet_bootstrap.cc:654] T 972763afd57847269fafadbdd2a5f3d6 P abe182b2afc64d908f692ef45eae1d51: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:21.294266 4180 raft_consensus.cc:504] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:28:21.296770 4148 raft_consensus.cc:2229] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:21.297317 4148 raft_consensus.cc:2258] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:21.299081 4148 ts_tablet_manager.cc:1905] T f58349edf0db40f98951e116fc2582b0 P abe182b2afc64d908f692ef45eae1d51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:21.308168 2401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 4 candidate_status { last_received { term: 3 index: 390 } } ignore_live_leader: false dest_uuid: "19a88ff0709d4266b933cc526577127f"
I20260927 23:28:21.312017 3167 tablet_bootstrap.cc:492] T 972763afd57847269fafadbdd2a5f3d6 P abe182b2afc64d908f692ef45eae1d51: No bootstrap required, opened a new log
I20260927 23:28:21.312446 3167 ts_tablet_manager.cc:1397] T 972763afd57847269fafadbdd2a5f3d6 P abe182b2afc64d908f692ef45eae1d51: Time spent bootstrapping tablet: real 0.041s user 0.009s sys 0.000s
I20260927 23:28:21.313968 3820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bee21c28e51643189178a3c851672546" candidate_term: 4 candidate_status { last_received { term: 3 index: 390 } } ignore_live_leader: false dest_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150"
I20260927 23:28:21.314504 3820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 3 FOLLOWER]: Advancing to term 4
I20260927 23:28:21.314491 3167 raft_consensus.cc:348] T 972763afd57847269fafadbdd2a5f3d6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.315094 3167 raft_consensus.cc:374] T 972763afd57847269fafadbdd2a5f3d6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:21.315313 3167 raft_consensus.cc:729] T 972763afd57847269fafadbdd2a5f3d6 P abe182b2afc64d908f692ef45eae1d51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe182b2afc64d908f692ef45eae1d51, State: Initialized, Role: FOLLOWER
I20260927 23:28:21.317061 4180 leader_election.cc:288] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 election: Requested vote from peers 19a88ff0709d4266b933cc526577127f (127.2.57.61:34987), 0cdd64d48a684d5ebccc2e2e8a4e3150 (127.2.57.60:41571)
I20260927 23:28:21.323426 4150 ts_tablet_manager.cc:1918] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:28:21.325793 3820 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0cdd64d48a684d5ebccc2e2e8a4e3150 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bee21c28e51643189178a3c851672546 in term 4.
I20260927 23:28:21.326841 2303 leader_election.cc:302] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0cdd64d48a684d5ebccc2e2e8a4e3150, bee21c28e51643189178a3c851672546; no voters: 19a88ff0709d4266b933cc526577127f
I20260927 23:28:21.328470 4150 log.cc:1196] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_0.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/8cd0972f635d4979abf5fd30e9b02b1d
I20260927 23:28:21.328848 4180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 4 FOLLOWER]: Leader election won for term 4
I20260927 23:28:21.329792 4150 ts_tablet_manager.cc:1939] T 8cd0972f635d4979abf5fd30e9b02b1d P de216a4a8e4b4e1b857d8647a3b05f6d: Deleting consensus metadata
I20260927 23:28:21.332672 2276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu with pid 2731
W20260927 23:28:21.327809 2907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 74929.5us
I20260927 23:28:21.338830 2378 catalog_manager.cc:4841] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): tablet 8cd0972f635d4979abf5fd30e9b02b1d (table default.Table_813e5d484f8f462b86e9136a57eebacd [id=b215370f7b594ab28c3e2a34de2a6f16]) successfully deleted
W20260927 23:28:21.340724 2378 catalog_manager.cc:4808] TS de216a4a8e4b4e1b857d8647a3b05f6d (127.2.57.3:33209): delete failed for tablet a1890852164d492db3540a97847888fd because the tablet was not found. No further retry: Not found: Tablet not found: a1890852164d492db3540a97847888fd
I20260927 23:28:21.344070 4180 raft_consensus.cc:686] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 4 LEADER]: Becoming Leader. State: Replica: bee21c28e51643189178a3c851672546, State: Running, Role: LEADER
I20260927 23:28:21.345089 4180 consensus_queue.cc:237] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 390, Committed index: 390, Last appended: 3.390, Last appended by leader: 390, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
W20260927 23:28:21.345698 2907 leader_election.cc:334] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Network error: Client connection negotiation failed: client connection to 127.2.57.1:35657: connect: Connection refused (error 111)
W20260927 23:28:21.346407 2907 leader_election.cc:334] T 119a1338b04740d0bbf291c8fbe34992 P de216a4a8e4b4e1b857d8647a3b05f6d [CANDIDATE]: Term 1 election: RPC error from VoteRequest() call to peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Network error: Client connection negotiation failed: client connection to 127.2.57.1:35657: connect: Connection refused (error 111)
W20260927 23:28:21.344383 2380 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.2.57.1:35657, user_credentials={real_user=slave}} blocked reactor thread for 85163.8us
W20260927 23:28:21.346941 2907 consensus_peers.cc:487] T 2d0d8a82c5234880ab8396b1777e36c8 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.1:35657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:21.347266 2907 consensus_peers.cc:487] T 5c87d497da884544920b3be9c78daccf P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.1:35657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:21.347597 2907 consensus_peers.cc:487] T 40bce5362d2541329b253716bbd99e07 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.1:35657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:21.347905 2907 consensus_peers.cc:487] T 56766ed12a7c4a668a7e08fc1ad39d2f P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.1:35657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:21.348223 2907 consensus_peers.cc:487] T 371842420a6048fca3cbc02d18f74ffc P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.1:35657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:21.348806 2907 consensus_peers.cc:487] T fbfa55a66b8042ed8592fbff31e92d55 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.1:35657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:21.352352 2907 consensus_peers.cc:487] T 7f530db3b52944f6a283ee9e2bf66828 P de216a4a8e4b4e1b857d8647a3b05f6d -> Peer 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): Couldn't send request to peer 5ce671f028a34ea4b8f0520c2689c9a1. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.1:35657: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:21.359881 3072 tablet_bootstrap.cc:654] T a8df6d0652634def9c8946c346dea702 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:21.366631 4193 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bee21c28e51643189178a3c851672546. Latest consensus state: current_term: 4 leader_uuid: "bee21c28e51643189178a3c851672546" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } } }
I20260927 23:28:21.367864 2326 raft_consensus.cc:1226] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [term 4 LEADER]: Rejecting Update request from peer 19a88ff0709d4266b933cc526577127f for earlier term 3. Current term is 4. Ops: [3.391-3.391]
I20260927 23:28:21.371485 4193 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee21c28e51643189178a3c851672546 [sys.catalog]: This master's current role is: LEADER
I20260927 23:28:21.371534 4184 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 }, Status: INVALID_TERM, Last received: 3.390, Next index: 391, Last known committed idx: 390, Time since last communication: 0.001s
I20260927 23:28:21.372205 4184 consensus_queue.cc:1225] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [LEADER]: Peer bee21c28e51643189178a3c851672546 log is divergent from this leader: its last log entry 4.391 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 390
I20260927 23:28:21.372733 4184 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 3 LEADER]: Stepping down as leader of term 3
I20260927 23:28:21.373011 4184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 19a88ff0709d4266b933cc526577127f, State: Running, Role: LEADER
I20260927 23:28:21.373688 4184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 390, Committed index: 390, Last appended: 3.391, Last appended by leader: 391, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bee21c28e51643189178a3c851672546" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 38773 } } peers { permanent_uuid: "19a88ff0709d4266b933cc526577127f" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 34987 } } peers { permanent_uuid: "0cdd64d48a684d5ebccc2e2e8a4e3150" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 41571 } }
I20260927 23:28:17.464779 3821 rpcz_store.cc:276] 0927 23:28:14.027541 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:14.027709 (+ 168us) service_pool.cc:227] Handling call
0927 23:28:14.436101 (+408392us) raft_consensus.cc:1457] Updating replica for 310 ops
0927 23:28:14.515534 (+ 79433us) raft_consensus.cc:1520] Early marking committed up to index 0
0927 23:28:14.515585 (+ 51us) raft_consensus.cc:1525] Triggering prepare for 310 ops
0927 23:28:14.524422 (+ 8837us) write_op.cc:270] Start()
0927 23:28:14.524610 (+ 188us) write_op.cc:276] Timestamp: P: 1790551674462110 usec, L: 0
0927 23:28:14.558902 (+ 34292us) write_op.cc:270] Start()
0927 23:28:14.559011 (+ 109us) write_op.cc:276] Timestamp: P: 1790551674467380 usec, L: 0
0927 23:28:14.559598 (+ 587us) write_op.cc:270] Start()
0927 23:28:14.559711 (+ 113us) write_op.cc:276] Timestamp: P: 1790551674469556 usec, L: 0
0927 23:28:14.564509 (+ 4798us) write_op.cc:270] Start()
0927 23:28:14.564605 (+ 96us) write_op.cc:276] Timestamp: P: 1790551674473124 usec, L: 0
0927 23:28:14.565147 (+ 542us) write_op.cc:270] Start()
0927 23:28:14.565227 (+ 80us) write_op.cc:276] Timestamp: P: 1790551674519830 usec, L: 0
0927 23:28:14.565681 (+ 454us) write_op.cc:270] Start()
0927 23:28:14.565795 (+ 114us) write_op.cc:276] Timestamp: P: 1790551674528913 usec, L: 0
0927 23:28:14.566228 (+ 433us) write_op.cc:270] Start()
0927 23:28:14.566376 (+ 148us) write_op.cc:276] Timestamp: P: 1790551674531128 usec, L: 0
0927 23:28:14.566829 (+ 453us) write_op.cc:270] Start()
0927 23:28:14.566905 (+ 76us) write_op.cc:276] Timestamp: P: 1790551674533249 usec, L: 0
0927 23:28:14.582454 (+ 15549us) write_op.cc:270] Start()
0927 23:28:14.582568 (+ 114us) write_op.cc:276] Timestamp: P: 1790551674535659 usec, L: 0
0927 23:28:14.583181 (+ 613us) write_op.cc:270] Start()
0927 23:28:14.583263 (+ 82us) write_op.cc:276] Timestamp: P: 1790551674636825 usec, L: 0
0927 23:28:14.584517 (+ 1254us) write_op.cc:270] Start()
0927 23:28:14.584592 (+ 75us) write_op.cc:276] Timestamp: P: 1790551674712622 usec, L: 0
0927 23:28:14.585052 (+ 460us) write_op.cc:270] Start()
0927 23:28:14.585149 (+ 97us) write_op.cc:276] Timestamp: P: 1790551674734339 usec, L: 0
0927 23:28:14.585606 (+ 457us) write_op.cc:270] Start()
0927 23:28:14.585687 (+ 81us) write_op.cc:276] Timestamp: P: 1790551674757810 usec, L: 0
0927 23:28:14.586160 (+ 473us) write_op.cc:270] Start()
0927 23:28:14.586237 (+ 77us) write_op.cc:276] Timestamp: P: 1790551674805755 usec, L: 0
0927 23:28:14.586675 (+ 438us) write_op.cc:270] Start()
0927 23:28:14.586777 (+ 102us) write_op.cc:276] Timestamp: P: 1790551674846744 usec, L: 0
0927 23:28:14.587252 (+ 475us) write_op.cc:270] Start()
0927 23:28:14.587342 (+ 90us) write_op.cc:276] Timestamp: P: 1790551674921761 usec, L: 0
0927 23:28:14.613885 (+ 26543us) write_op.cc:270] Start()
0927 23:28:14.613991 (+ 106us) write_op.cc:276] Timestamp: P: 1790551674998045 usec, L: 0
0927 23:28:14.614553 (+ 562us) write_op.cc:270] Start()
0927 23:28:14.614643 (+ 90us) write_op.cc:276] Timestamp: P: 1790551675013820 usec, L: 0
0927 23:28:14.615101 (+ 458us) write_op.cc:270] Start()
0927 23:28:14.635469 (+ 20368us) write_op.cc:276] Timestamp: P: 1790551675020888 usec, L: 0
0927 23:28:14.636140 (+ 671us) write_op.cc:270] Start()
0927 23:28:14.636244 (+ 104us) write_op.cc:276] Timestamp: P: 1790551675024067 usec, L: 0
0927 23:28:14.636767 (+ 523us) write_op.cc:270] Start()
0927 23:28:14.636856 (+ 89us) write_op.cc:276] Timestamp: P: 1790551675026240 usec, L: 0
0927 23:28:14.637326 (+ 470us) write_op.cc:270] Start()
0927 23:28:14.637402 (+ 76us) write_op.cc:276] Timestamp: P: 1790551675035498 usec, L: 0
0927 23:28:14.637823 (+ 421us) write_op.cc:270] Start()
0927 23:28:14.637913 (+ 90us) write_op.cc:276] Timestamp: P: 1790551675102599 usec, L: 0
0927 23:28:14.638352 (+ 439us) write_op.cc:270] Start()
0927 23:28:14.638485 (+ 133us) write_op.cc:276] Timestamp: P: 1790551675143709 usec, L: 0
0927 23:28:14.639047 (+ 562us) write_op.cc:270] Start()
0927 23:28:14.639135 (+ 88us) write_op.cc:276] Timestamp: P: 1790551675157091 usec, L: 0
0927 23:28:14.639589 (+ 454us) write_op.cc:270] Start()
0927 23:28:14.639663 (+ 74us) write_op.cc:276] Timestamp: P: 1790551675304550 usec, L: 0
0927 23:28:14.640069 (+ 406us) write_op.cc:270] Start()
0927 23:28:14.640143 (+ 74us) write_op.cc:276] Timestamp: P: 1790551675307411 usec, L: 0
0927 23:28:14.653096 (+ 12953us) write_op.cc:270] Start()
0927 23:28:14.653207 (+ 111us) write_op.cc:276] Timestamp: P: 1790551675310051 usec, L: 0
0927 23:28:14.653759 (+ 552us) write_op.cc:270] Start()
0927 23:28:14.653841 (+ 82us) write_op.cc:276] Timestamp: P: 1790551675362346 usec, L: 0
0927 23:28:14.654301 (+ 460us) write_op.cc:270] Start()
0927 23:28:14.654385 (+ 84us) write_op.cc:276] Timestamp: P: 1790551675389124 usec, L: 0
0927 23:28:14.654803 (+ 418us) write_op.cc:270] Start()
0927 23:28:14.654878 (+ 75us) write_op.cc:276] Timestamp: P: 1790551675424413 usec, L: 0
0927 23:28:14.655520 (+ 642us) write_op.cc:270] Start()
0927 23:28:14.655986 (+ 466us) write_op.cc:276] Timestamp: P: 1790551675431454 usec, L: 0
0927 23:28:14.674449 (+ 18463us) write_op.cc:270] Start()
0927 23:28:14.674556 (+ 107us) write_op.cc:276] Timestamp: P: 1790551675484001 usec, L: 0
0927 23:28:14.675104 (+ 548us) write_op.cc:270] Start()
0927 23:28:14.675192 (+ 88us) write_op.cc:276] Timestamp: P: 1790551675509726 usec, L: 0
0927 23:28:14.675617 (+ 425us) write_op.cc:270] Start()
0927 23:28:14.675692 (+ 75us) write_op.cc:276] Timestamp: P: 1790551675572688 usec, L: 0
0927 23:28:14.676145 (+ 453us) write_op.cc:270] Start()
0927 23:28:14.676234 (+ 89us) write_op.cc:276] Timestamp: P: 1790551675643192 usec, L: 0
0927 23:28:14.676693 (+ 459us) write_op.cc:270] Start()
0927 23:28:14.676783 (+ 90us) write_op.cc:276] Timestamp: P: 1790551675648005 usec, L: 0
0927 23:28:14.677207 (+ 424us) write_op.cc:270] Start()
0927 23:28:14.677282 (+ 75us) write_op.cc:276] Timestamp: P: 1790551675744346 usec, L: 0
0927 23:28:14.677792 (+ 510us) write_op.cc:270] Start()
0927 23:28:14.677875 (+ 83us) write_op.cc:276] Timestamp: P: 1790551675780941 usec, L: 0
0927 23:28:14.678302 (+ 427us) write_op.cc:270] Start()
0927 23:28:14.678388 (+ 86us) write_op.cc:276] Timestamp: P: 1790551675792427 usec, L: 0
0927 23:28:14.679040 (+ 652us) write_op.cc:270] Start()
0927 23:28:14.679128 (+ 88us) write_op.cc:276] Timestamp: P: 1790551675813513 usec, L: 0
0927 23:28:14.679588 (+ 460us) write_op.cc:270] Start()
0927 23:28:14.679685 (+ 97us) write_op.cc:276] Timestamp: P: 1790551675827110 usec, L: 0
0927 23:28:14.680164 (+ 479us) write_op.cc:270] Start()
0927 23:28:14.680250 (+ 86us) write_op.cc:276] Timestamp: P: 1790551675950739 usec, L: 0
0927 23:28:14.703844 (+ 23594us) write_op.cc:270] Start()
0927 23:28:14.703958 (+ 114us) write_op.cc:276] Timestamp: P: 1790551676028802 usec, L: 0
0927 23:28:14.707135 (+ 3177us) write_op.cc:270] Start()
0927 23:28:14.707693 (+ 558us) write_op.cc:276] Timestamp: P: 1790551676163684 usec, L: 0
0927 23:28:14.708272 (+ 579us) write_op.cc:270] Start()
0927 23:28:14.708391 (+ 119us) write_op.cc:276] Timestamp: P: 1790551676275814 usec, L: 0
0927 23:28:14.708890 (+ 499us) write_op.cc:270] Start()
0927 23:28:14.708969 (+ 79us) write_op.cc:276] Timestamp: P: 1790551676278578 usec, L: 0
0927 23:28:14.709877 (+ 908us) write_op.cc:270] Start()
0927 23:28:14.710024 (+ 147us) write_op.cc:276] Timestamp: P: 1790551676432670 usec, L: 0
0927 23:28:14.710541 (+ 517us) write_op.cc:270] Start()
0927 23:28:14.710618 (+ 77us) write_op.cc:276] Timestamp: P: 1790551676479762 usec, L: 0
0927 23:28:14.711030 (+ 412us) write_op.cc:270] Start()
0927 23:28:14.711113 (+ 83us) write_op.cc:276] Timestamp: P: 1790551676499110 usec, L: 0
0927 23:28:14.732540 (+ 21427us) write_op.cc:270] Start()
0927 23:28:14.732677 (+ 137us) write_op.cc:276] Timestamp: P: 1790551676548481 usec, L: 0
0927 23:28:14.733371 (+ 694us) write_op.cc:270] Start()
0927 23:28:14.733462 (+ 91us) write_op.cc:276] Timestamp: P: 1790551676551301 usec, L: 0
0927 23:28:14.735625 (+ 2163us) write_op.cc:270] Start()
0927 23:28:14.735719 (+ 94us) write_op.cc:276] Timestamp: P: 1790551676620745 usec, L: 0
0927 23:28:14.752525 (+ 16806us) write_op.cc:270] Start()
0927 23:28:14.752646 (+ 121us) write_op.cc:276] Timestamp: P: 1790551676680152 usec, L: 0
0927 23:28:14.753276 (+ 630us) write_op.cc:270] Start()
0927 23:28:14.753364 (+ 88us) write_op.cc:276] Timestamp: P: 1790551676721900 usec, L: 0
0927 23:28:14.753863 (+ 499us) write_op.cc:270] Start()
0927 23:28:14.753944 (+ 81us) write_op.cc:276] Timestamp: P: 1790551676769187 usec, L: 0
0927 23:28:14.754386 (+ 442us) write_op.cc:270] Start()
0927 23:28:14.754472 (+ 86us) write_op.cc:276] Timestamp: P: 1790551676828156 usec, L: 0
0927 23:28:14.754894 (+ 422us) write_op.cc:270] Start()
0927 23:28:14.754967 (+ 73us) write_op.cc:276] Timestamp: P: 1790551676882249 usec, L: 0
0927 23:28:14.755436 (+ 469us) write_op.cc:270] Start()
0927 23:28:14.755517 (+ 81us) write_op.cc:276] Timestamp: P: 1790551676918973 usec, L: 0
0927 23:28:14.755947 (+ 430us) write_op.cc:270] Start()
0927 23:28:14.756107 (+ 160us) write_op.cc:276] Timestamp: P: 1790551676926725 usec, L: 0
0927 23:28:14.783757 (+ 27650us) write_op.cc:270] Start()
0927 23:28:14.783869 (+ 112us) write_op.cc:276] Timestamp: P: 1790551676928764 usec, L: 0
0927 23:28:14.787699 (+ 3830us) write_op.cc:270] Start()
0927 23:28:14.787821 (+ 122us) write_op.cc:276] Timestamp: P: 1790551677054136 usec, L: 0
0927 23:28:14.788404 (+ 583us) write_op.cc:270] Start()
0927 23:28:14.788489 (+ 85us) write_op.cc:276] Timestamp: P: 1790551677119764 usec, L: 0
0927 23:28:14.788941 (+ 452us) write_op.cc:270] Start()
0927 23:28:14.789024 (+ 83us) write_op.cc:276] Timestamp: P: 1790551677167983 usec, L: 0
0927 23:28:14.789732 (+ 708us) write_op.cc:270] Start()
0927 23:28:14.789810 (+ 78us) write_op.cc:276] Timestamp: P: 1790551677169931 usec, L: 0
0927 23:28:14.790268 (+ 458us) write_op.cc:270] Start()
0927 23:28:14.790342 (+ 74us) write_op.cc:276] Timestamp: P: 1790551677201878 usec, L: 0
0927 23:28:14.790783 (+ 441us) write_op.cc:270] Start()
0927 23:28:14.790859 (+ 76us) write_op.cc:276] Timestamp: P: 1790551677239026 usec, L: 0
0927 23:28:14.791304 (+ 445us) write_op.cc:270] Start()
0927 23:28:14.791380 (+ 76us) write_op.cc:276] Timestamp: P: 1790551677349266 usec, L: 0
0927 23:28:14.813093 (+ 21713us) write_op.cc:270] Start()
0927 23:28:14.813229 (+ 136us) write_op.cc:276] Timestamp: P: 1790551677351214 usec, L: 0
0927 23:28:14.813787 (+ 558us) write_op.cc:270] Start()
0927 23:28:14.813876 (+ 89us) write_op.cc:276] Timestamp: P: 1790551677453960 usec, L: 0
0927 23:28:14.814355 (+ 479us) write_op.cc:270] Start()
0927 23:28:14.814434 (+ 79us) write_op.cc:276] Timestamp: P: 1790551677570822 usec, L: 0
0927 23:28:14.814882 (+ 448us) write_op.cc:270] Start()
0927 23:28:14.814964 (+ 82us) write_op.cc:276] Timestamp: P: 1790551677593514 usec, L: 0
0927 23:28:14.815505 (+ 541us) write_op.cc:270] Start()
0927 23:28:14.815589 (+ 84us) write_op.cc:276] Timestamp: P: 1790551677595528 usec, L: 0
0927 23:28:14.816032 (+ 443us) write_op.cc:270] Start()
0927 23:28:14.816106 (+ 74us) write_op.cc:276] Timestamp: P: 1790551677614091 usec, L: 0
0927 23:28:14.816594 (+ 488us) write_op.cc:270] Start()
0927 23:28:14.816690 (+ 96us) write_op.cc:276] Timestamp: P: 1790551677654728 usec, L: 0
0927 23:28:14.817129 (+ 439us) write_op.cc:270] Start()
0927 23:28:14.817212 (+ 83us) write_op.cc:276] Timestamp: P: 1790551677769836 usec, L: 0
0927 23:28:14.818531 (+ 1319us) write_op.cc:270] Start()
0927 23:28:14.818612 (+ 81us) write_op.cc:276] Timestamp: P: 1790551677774707 usec, L: 0
0927 23:28:14.819048 (+ 436us) write_op.cc:270] Start()
0927 23:28:14.819123 (+ 75us) write_op.cc:276] Timestamp: P: 1790551677777993 usec, L: 0
0927 23:28:14.819550 (+ 427us) write_op.cc:270] Start()
0927 23:28:14.819665 (+ 115us) write_op.cc:276] Timestamp: P: 1790551677780087 usec, L: 0
0927 23:28:14.829526 (+ 9861us) write_op.cc:270] Start()
0927 23:28:14.829637 (+ 111us) write_op.cc:276] Timestamp: P: 1790551677884249 usec, L: 0
0927 23:28:14.834079 (+ 4442us) write_op.cc:270] Start()
0927 23:28:14.834237 (+ 158us) write_op.cc:276] Timestamp: P: 1790551677886571 usec, L: 0
0927 23:28:14.834848 (+ 611us) write_op.cc:270] Start()
0927 23:28:14.834935 (+ 87us) write_op.cc:276] Timestamp: P: 1790551677928624 usec, L: 0
0927 23:28:14.835550 (+ 615us) write_op.cc:270] Start()
0927 23:28:14.835642 (+ 92us) write_op.cc:276] Timestamp: P: 1790551677953181 usec, L: 0
0927 23:28:14.836105 (+ 463us) write_op.cc:270] Start()
0927 23:28:14.836193 (+ 88us) write_op.cc:276] Timestamp: P: 1790551677984891 usec, L: 0
0927 23:28:14.837520 (+ 1327us) write_op.cc:270] Start()
0927 23:28:14.837603 (+ 83us) write_op.cc:276] Timestamp: P: 1790551678073622 usec, L: 0
0927 23:28:14.838027 (+ 424us) write_op.cc:270] Start()
0927 23:28:14.838100 (+ 73us) write_op.cc:276] Timestamp: P: 1790551678095611 usec, L: 0
0927 23:28:14.838511 (+ 411us) write_op.cc:270] Start()
0927 23:28:14.838593 (+ 82us) write_op.cc:276] Timestamp: P: 1790551678128886 usec, L: 0
0927 23:28:14.856695 (+ 18102us) write_op.cc:270] Start()
0927 23:28:14.856818 (+ 123us) write_op.cc:276] Timestamp: P: 1790551678165160 usec, L: 0
0927 23:28:14.857464 (+ 646us) write_op.cc:270] Start()
0927 23:28:14.857546 (+ 82us) write_op.cc:276] Timestamp: P: 1790551678187146 usec, L: 0
0927 23:28:14.857984 (+ 438us) write_op.cc:270] Start()
0927 23:28:14.858070 (+ 86us) write_op.cc:276] Timestamp: P: 1790551678229445 usec, L: 0
0927 23:28:14.860489 (+ 2419us) write_op.cc:270] Start()
0927 23:28:14.860576 (+ 87us) write_op.cc:276] Timestamp: P: 1790551678251508 usec, L: 0
0927 23:28:14.861034 (+ 458us) write_op.cc:270] Start()
0927 23:28:14.861114 (+ 80us) write_op.cc:276] Timestamp: P: 1790551678322133 usec, L: 0
0927 23:28:14.861542 (+ 428us) write_op.cc:270] Start()
0927 23:28:14.861616 (+ 74us) write_op.cc:276] Timestamp: P: 1790551678501550 usec, L: 0
0927 23:28:14.862055 (+ 439us) write_op.cc:270] Start()
0927 23:28:14.862136 (+ 81us) write_op.cc:276] Timestamp: P: 1790551678605780 usec, L: 0
0927 23:28:14.862575 (+ 439us) write_op.cc:270] Start()
0927 23:28:14.862648 (+ 73us) write_op.cc:276] Timestamp: P: 1790551678689916 usec, L: 0
0927 23:28:14.863284 (+ 636us) write_op.cc:270] Start()
0927 23:28:14.863388 (+ 104us) write_op.cc:276] Timestamp: P: 1790551678691948 usec, L: 0
0927 23:28:14.868784 (+ 5396us) write_op.cc:270] Start()
0927 23:28:14.868882 (+ 98us) write_op.cc:276] Timestamp: P: 1790551678872631 usec, L: 0
0927 23:28:14.871162 (+ 2280us) write_op.cc:270] Start()
0927 23:28:14.871389 (+ 227us) write_op.cc:276] Timestamp: P: 1790551678891662 usec, L: 0
0927 23:28:14.872027 (+ 638us) write_op.cc:270] Start()
0927 23:28:14.872105 (+ 78us) write_op.cc:276] Timestamp: P: 1790551678893824 usec, L: 0
0927 23:28:14.880692 (+ 8587us) write_op.cc:270] Start()
0927 23:28:14.880788 (+ 96us) write_op.cc:276] Timestamp: P: 1790551678917096 usec, L: 0
0927 23:28:14.881325 (+ 537us) write_op.cc:270] Start()
0927 23:28:14.881409 (+ 84us) write_op.cc:276] Timestamp: P: 1790551679053041 usec, L: 0
0927 23:28:14.881840 (+ 431us) write_op.cc:270] Start()
0927 23:28:14.881919 (+ 79us) write_op.cc:276] Timestamp: P: 1790551679055371 usec, L: 0
0927 23:28:14.882346 (+ 427us) write_op.cc:270] Start()
0927 23:28:14.882423 (+ 77us) write_op.cc:276] Timestamp: P: 1790551679110456 usec, L: 0
0927 23:28:14.882840 (+ 417us) write_op.cc:270] Start()
0927 23:28:14.882909 (+ 69us) write_op.cc:276] Timestamp: P: 1790551679150275 usec, L: 0
0927 23:28:14.883355 (+ 446us) write_op.cc:270] Start()
0927 23:28:14.883434 (+ 79us) write_op.cc:276] Timestamp: P: 1790551679211762 usec, L: 0
0927 23:28:14.883850 (+ 416us) write_op.cc:270] Start()
0927 23:28:14.883931 (+ 81us) write_op.cc:276] Timestamp: P: 1790551679325659 usec, L: 0
0927 23:28:14.884389 (+ 458us) write_op.cc:270] Start()
0927 23:28:14.884466 (+ 77us) write_op.cc:276] Timestamp: P: 1790551679334882 usec, L: 0
0927 23:28:14.884878 (+ 412us) write_op.cc:270] Start()
0927 23:28:14.884953 (+ 75us) write_op.cc:276] Timestamp: P: 1790551679341404 usec, L: 0
0927 23:28:14.885396 (+ 443us) write_op.cc:270] Start()
0927 23:28:14.885475 (+ 79us) write_op.cc:276] Timestamp: P: 1790551679494168 usec, L: 0
0927 23:28:14.885905 (+ 430us) write_op.cc:270] Start()
0927 23:28:14.885996 (+ 91us) write_op.cc:276] Timestamp: P: 1790551679589861 usec, L: 0
0927 23:28:14.886487 (+ 491us) write_op.cc:270] Start()
0927 23:28:14.886565 (+ 78us) write_op.cc:276] Timestamp: P: 1790551679626381 usec, L: 0
0927 23:28:14.886997 (+ 432us) write_op.cc:270] Start()
0927 23:28:14.887083 (+ 86us) write_op.cc:276] Timestamp: P: 1790551679678005 usec, L: 0
0927 23:28:14.887580 (+ 497us) write_op.cc:270] Start()
0927 23:28:14.887659 (+ 79us) write_op.cc:276] Timestamp: P: 1790551679699701 usec, L: 0
0927 23:28:14.888094 (+ 435us) write_op.cc:270] Start()
0927 23:28:14.888169 (+ 75us) write_op.cc:276] Timestamp: P: 1790551679723110 usec, L: 0
0927 23:28:14.913065 (+ 24896us) write_op.cc:270] Start()
0927 23:28:14.913175 (+ 110us) write_op.cc:276] Timestamp: P: 1790551679727473 usec, L: 0
0927 23:28:14.913714 (+ 539us) write_op.cc:270] Start()
0927 23:28:14.913791 (+ 77us) write_op.cc:276] Timestamp: P: 1790551679867573 usec, L: 0
0927 23:28:14.914219 (+ 428us) write_op.cc:270] Start()
0927 23:28:14.914299 (+ 80us) write_op.cc:276] Timestamp: P: 1790551680035517 usec, L: 0
0927 23:28:14.914734 (+ 435us) write_op.cc:270] Start()
0927 23:28:14.914813 (+ 79us) write_op.cc:276] Timestamp: P: 1790551680386896 usec, L: 0
0927 23:28:14.915233 (+ 420us) write_op.cc:270] Start()
0927 23:28:14.915326 (+ 93us) write_op.cc:276] Timestamp: P: 1790551680554644 usec, L: 0
0927 23:28:14.920336 (+ 5010us) write_op.cc:270] Start()
0927 23:28:14.920428 (+ 92us) write_op.cc:276] Timestamp: P: 1790551680810914 usec, L: 0
0927 23:28:14.921002 (+ 574us) write_op.cc:270] Start()
0927 23:28:14.921079 (+ 77us) write_op.cc:276] Timestamp: P: 1790551681139036 usec, L: 0
0927 23:28:14.922054 (+ 975us) write_op.cc:270] Start()
0927 23:28:14.922142 (+ 88us) write_op.cc:276] Timestamp: P: 1790551681214997 usec, L: 0
0927 23:28:14.923452 (+ 1310us) write_op.cc:270] Start()
0927 23:28:14.923526 (+ 74us) write_op.cc:276] Timestamp: P: 1790551681237555 usec, L: 0
0927 23:28:14.923932 (+ 406us) write_op.cc:270] Start()
0927 23:28:14.924009 (+ 77us) write_op.cc:276] Timestamp: P: 1790551681630924 usec, L: 0
0927 23:28:14.924485 (+ 476us) write_op.cc:270] Start()
0927 23:28:14.924560 (+ 75us) write_op.cc:276] Timestamp: P: 1790551681934333 usec, L: 0
0927 23:28:14.924981 (+ 421us) write_op.cc:270] Start()
0927 23:28:14.925060 (+ 79us) write_op.cc:276] Timestamp: P: 1790551682108702 usec, L: 0
0927 23:28:14.934653 (+ 9593us) write_op.cc:270] Start()
0927 23:28:14.934760 (+ 107us) write_op.cc:276] Timestamp: P: 1790551682143127 usec, L: 0
0927 23:28:14.935326 (+ 566us) write_op.cc:270] Start()
0927 23:28:14.935704 (+ 378us) write_op.cc:276] Timestamp: P: 1790551682146775 usec, L: 0
0927 23:28:14.936470 (+ 766us) write_op.cc:270] Start()
0927 23:28:14.936549 (+ 79us) write_op.cc:276] Timestamp: P: 1790551682149763 usec, L: 0
0927 23:28:14.937020 (+ 471us) write_op.cc:270] Start()
0927 23:28:14.941177 (+ 4157us) write_op.cc:276] Timestamp: P: 1790551682206904 usec, L: 0
0927 23:28:14.941735 (+ 558us) write_op.cc:270] Start()
0927 23:28:14.941902 (+ 167us) write_op.cc:276] Timestamp: P: 1790551682346581 usec, L: 0
0927 23:28:14.942506 (+ 604us) write_op.cc:270] Start()
0927 23:28:14.942634 (+ 128us) write_op.cc:276] Timestamp: P: 1790551682354106 usec, L: 0
0927 23:28:14.943091 (+ 457us) write_op.cc:270] Start()
0927 23:28:14.943179 (+ 88us) write_op.cc:276] Timestamp: P: 1790551682357752 usec, L: 0
0927 23:28:14.948196 (+ 5017us) write_op.cc:270] Start()
0927 23:28:14.948289 (+ 93us) write_op.cc:276] Timestamp: P: 1790551682365548 usec, L: 0
0927 23:28:14.948779 (+ 490us) write_op.cc:270] Start()
0927 23:28:14.948845 (+ 66us) write_op.cc:276] Timestamp: P: 1790551682414993 usec, L: 0
0927 23:28:14.949279 (+ 434us) write_op.cc:270] Start()
0927 23:28:14.949351 (+ 72us) write_op.cc:276] Timestamp: P: 1790551682424161 usec, L: 0
0927 23:28:14.949742 (+ 391us) write_op.cc:270] Start()
0927 23:28:14.949805 (+ 63us) write_op.cc:276] Timestamp: P: 1790551682470266 usec, L: 0
0927 23:28:14.950183 (+ 378us) write_op.cc:270] Start()
0927 23:28:14.950260 (+ 77us) write_op.cc:276] Timestamp: P: 1790551682599215 usec, L: 0
0927 23:28:14.950679 (+ 419us) write_op.cc:270] Start()
0927 23:28:14.950752 (+ 73us) write_op.cc:276] Timestamp: P: 1790551682633266 usec, L: 0
0927 23:28:14.951146 (+ 394us) write_op.cc:270] Start()
0927 23:28:14.951212 (+ 66us) write_op.cc:276] Timestamp: P: 1790551682663034 usec, L: 0
0927 23:28:14.951595 (+ 383us) write_op.cc:270] Start()
0927 23:28:14.951664 (+ 69us) write_op.cc:276] Timestamp: P: 1790551682681146 usec, L: 0
0927 23:28:14.952054 (+ 390us) write_op.cc:270] Start()
0927 23:28:14.952128 (+ 74us) write_op.cc:276] Timestamp: P: 1790551682758001 usec, L: 0
0927 23:28:14.952601 (+ 473us) write_op.cc:270] Start()
0927 23:28:14.952676 (+ 75us) write_op.cc:276] Timestamp: P: 1790551682784309 usec, L: 0
0927 23:28:14.953074 (+ 398us) write_op.cc:270] Start()
0927 23:28:14.953148 (+ 74us) write_op.cc:276] Timestamp: P: 1790551682799598 usec, L: 0
0927 23:28:14.953530 (+ 382us) write_op.cc:270] Start()
0927 23:28:14.953598 (+ 68us) write_op.cc:276] Timestamp: P: 1790551682926405 usec, L: 0
0927 23:28:14.953984 (+ 386us) write_op.cc:270] Start()
0927 23:28:14.954049 (+ 65us) write_op.cc:276] Timestamp: P: 1790551682954092 usec, L: 0
0927 23:28:14.954453 (+ 404us) write_op.cc:270] Start()
0927 23:28:14.954530 (+ 77us) write_op.cc:276] Timestamp: P: 1790551683006039 usec, L: 0
0927 23:28:14.954918 (+ 388us) write_op.cc:270] Start()
0927 23:28:14.954991 (+ 73us) write_op.cc:276] Timestamp: P: 1790551683010206 usec, L: 0
0927 23:28:14.955439 (+ 448us) write_op.cc:270] Start()
0927 23:28:14.955509 (+ 70us) write_op.cc:276] Timestamp: P: 1790551683069784 usec, L: 0
0927 23:28:14.955924 (+ 415us) write_op.cc:270] Start()
0927 23:28:14.956019 (+ 95us) write_op.cc:276] Timestamp: P: 1790551683187613 usec, L: 0
0927 23:28:14.956457 (+ 438us) write_op.cc:270] Start()
0927 23:28:14.956527 (+ 70us) write_op.cc:276] Timestamp: P: 1790551683196341 usec, L: 0
0927 23:28:14.956915 (+ 388us) write_op.cc:270] Start()
0927 23:28:14.981462 (+ 24547us) write_op.cc:276] Timestamp: P: 1790551683232083 usec, L: 0
0927 23:28:14.982263 (+ 801us) write_op.cc:270] Start()
0927 23:28:14.982358 (+ 95us) write_op.cc:276] Timestamp: P: 1790551683277519 usec, L: 0
0927 23:28:14.982803 (+ 445us) write_op.cc:270] Start()
0927 23:28:14.982873 (+ 70us) write_op.cc:276] Timestamp: P: 1790551683354960 usec, L: 0
0927 23:28:14.983311 (+ 438us) write_op.cc:270] Start()
0927 23:28:14.983378 (+ 67us) write_op.cc:276] Timestamp: P: 1790551683363862 usec, L: 0
0927 23:28:14.983768 (+ 390us) write_op.cc:270] Start()
0927 23:28:14.983835 (+ 67us) write_op.cc:276] Timestamp: P: 1790551683380915 usec, L: 0
0927 23:28:14.984203 (+ 368us) write_op.cc:270] Start()
0927 23:28:14.984278 (+ 75us) write_op.cc:276] Timestamp: P: 1790551683450022 usec, L: 0
0927 23:28:14.984681 (+ 403us) write_op.cc:270] Start()
0927 23:28:14.984750 (+ 69us) write_op.cc:276] Timestamp: P: 1790551683465028 usec, L: 0
0927 23:28:14.985163 (+ 413us) write_op.cc:270] Start()
0927 23:28:14.985243 (+ 80us) write_op.cc:276] Timestamp: P: 1790551683654429 usec, L: 0
0927 23:28:14.985650 (+ 407us) write_op.cc:270] Start()
0927 23:28:14.985726 (+ 76us) write_op.cc:276] Timestamp: P: 1790551683656801 usec, L: 0
0927 23:28:14.986129 (+ 403us) write_op.cc:270] Start()
0927 23:28:14.986198 (+ 69us) write_op.cc:276] Timestamp: P: 1790551683676811 usec, L: 0
0927 23:28:14.986638 (+ 440us) write_op.cc:270] Start()
0927 23:28:14.986714 (+ 76us) write_op.cc:276] Timestamp: P: 1790551683680210 usec, L: 0
0927 23:28:14.987086 (+ 372us) write_op.cc:270] Start()
0927 23:28:14.987148 (+ 62us) write_op.cc:276] Timestamp: P: 1790551683683162 usec, L: 0
0927 23:28:14.987538 (+ 390us) write_op.cc:270] Start()
0927 23:28:14.987604 (+ 66us) write_op.cc:276] Timestamp: P: 1790551683835245 usec, L: 0
0927 23:28:14.987999 (+ 395us) write_op.cc:270] Start()
0927 23:28:14.988068 (+ 69us) write_op.cc:276] Timestamp: P: 1790551683889587 usec, L: 0
0927 23:28:14.988506 (+ 438us) write_op.cc:270] Start()
0927 23:28:14.988577 (+ 71us) write_op.cc:276] Timestamp: P: 1790551683916979 usec, L: 0
0927 23:28:14.989049 (+ 472us) write_op.cc:270] Start()
0927 23:28:14.989115 (+ 66us) write_op.cc:276] Timestamp: P: 1790551683937977 usec, L: 0
0927 23:28:14.989555 (+ 440us) write_op.cc:270] Start()
0927 23:28:14.989632 (+ 77us) write_op.cc:276] Timestamp: P: 1790551684036737 usec, L: 0
0927 23:28:14.990036 (+ 404us) write_op.cc:270] Start()
0927 23:28:14.990105 (+ 69us) write_op.cc:276] Timestamp: P: 1790551684063795 usec, L: 0
0927 23:28:14.990529 (+ 424us) write_op.cc:270] Start()
0927 23:28:14.990593 (+ 64us) write_op.cc:276] Timestamp: P: 1790551684075745 usec, L: 0
0927 23:28:14.990982 (+ 389us) write_op.cc:270] Start()
0927 23:28:14.991045 (+ 63us) write_op.cc:276] Timestamp: P: 1790551684152818 usec, L: 0
0927 23:28:14.991519 (+ 474us) write_op.cc:270] Start()
0927 23:28:14.991590 (+ 71us) write_op.cc:276] Timestamp: P: 1790551684191109 usec, L: 0
0927 23:28:14.991968 (+ 378us) write_op.cc:270] Start()
0927 23:28:14.992028 (+ 60us) write_op.cc:276] Timestamp: P: 1790551684243349 usec, L: 0
0927 23:28:14.992538 (+ 510us) write_op.cc:270] Start()
0927 23:28:14.992622 (+ 84us) write_op.cc:276] Timestamp: P: 1790551684284929 usec, L: 0
0927 23:28:14.993002 (+ 380us) write_op.cc:270] Start()
0927 23:28:14.993067 (+ 65us) write_op.cc:276] Timestamp: P: 1790551684385595 usec, L: 0
0927 23:28:14.993483 (+ 416us) write_op.cc:270] Start()
0927 23:28:14.994718 (+ 1235us) write_op.cc:276] Timestamp: P: 1790551684390335 usec, L: 0
0927 23:28:14.995145 (+ 427us) write_op.cc:270] Start()
0927 23:28:14.995214 (+ 69us) write_op.cc:276] Timestamp: P: 1790551684436356 usec, L: 0
0927 23:28:14.995638 (+ 424us) write_op.cc:270] Start()
0927 23:28:14.995705 (+ 67us) write_op.cc:276] Timestamp: P: 1790551684439345 usec, L: 0
0927 23:28:15.003013 (+ 7308us) write_op.cc:270] Start()
0927 23:28:15.003103 (+ 90us) write_op.cc:276] Timestamp: P: 1790551684500906 usec, L: 0
0927 23:28:15.003598 (+ 495us) write_op.cc:270] Start()
0927 23:28:15.003669 (+ 71us) write_op.cc:276] Timestamp: P: 1790551684538316 usec, L: 0
0927 23:28:15.004037 (+ 368us) write_op.cc:270] Start()
0927 23:28:15.004104 (+ 67us) write_op.cc:276] Timestamp: P: 1790551684678375 usec, L: 0
0927 23:28:15.004549 (+ 445us) write_op.cc:270] Start()
0927 23:28:15.004614 (+ 65us) write_op.cc:276] Timestamp: P: 1790551684694955 usec, L: 0
0927 23:28:15.005023 (+ 409us) write_op.cc:270] Start()
0927 23:28:15.005098 (+ 75us) write_op.cc:276] Timestamp: P: 1790551684888416 usec, L: 0
0927 23:28:15.005507 (+ 409us) write_op.cc:270] Start()
0927 23:28:15.005608 (+ 101us) write_op.cc:276] Timestamp: P: 1790551684891601 usec, L: 0
0927 23:28:15.007253 (+ 1645us) write_op.cc:270] Start()
0927 23:28:15.007331 (+ 78us) write_op.cc:276] Timestamp: P: 1790551684929960 usec, L: 0
0927 23:28:15.007745 (+ 414us) write_op.cc:270] Start()
0927 23:28:15.007809 (+ 64us) write_op.cc:276] Timestamp: P: 1790551685157342 usec, L: 0
0927 23:28:15.008192 (+ 383us) write_op.cc:270] Start()
0927 23:28:15.008281 (+ 89us) write_op.cc:276] Timestamp: P: 1790551685318391 usec, L: 0
0927 23:28:15.012672 (+ 4391us) write_op.cc:270] Start()
0927 23:28:15.012754 (+ 82us) write_op.cc:276] Timestamp: P: 1790551685388974 usec, L: 0
0927 23:28:15.015816 (+ 3062us) write_op.cc:270] Start()
0927 23:28:15.015905 (+ 89us) write_op.cc:276] Timestamp: P: 1790551685522553 usec, L: 0
0927 23:28:15.016421 (+ 516us) write_op.cc:270] Start()
0927 23:28:15.022713 (+ 6292us) write_op.cc:276] Timestamp: P: 1790551685591595 usec, L: 0
0927 23:28:15.024737 (+ 2024us) write_op.cc:270] Start()
0927 23:28:15.024830 (+ 93us) write_op.cc:276] Timestamp: P: 1790551685594561 usec, L: 0
0927 23:28:15.025415 (+ 585us) write_op.cc:270] Start()
0927 23:28:15.025486 (+ 71us) write_op.cc:276] Timestamp: P: 1790551685662041 usec, L: 0
0927 23:28:15.045890 (+ 20404us) write_op.cc:270] Start()
0927 23:28:15.045990 (+ 100us) write_op.cc:276] Timestamp: P: 1790551685710661 usec, L: 0
0927 23:28:15.046555 (+ 565us) write_op.cc:270] Start()
0927 23:28:15.046627 (+ 72us) write_op.cc:276] Timestamp: P: 1790551685754956 usec, L: 0
0927 23:28:15.047037 (+ 410us) write_op.cc:270] Start()
0927 23:28:15.047100 (+ 63us) write_op.cc:276] Timestamp: P: 1790551685797659 usec, L: 0
0927 23:28:15.047492 (+ 392us) write_op.cc:270] Start()
0927 23:28:15.047561 (+ 69us) write_op.cc:276] Timestamp: P: 1790551685834257 usec, L: 0
0927 23:28:15.047971 (+ 410us) write_op.cc:270] Start()
0927 23:28:15.048036 (+ 65us) write_op.cc:276] Timestamp: P: 17905516
I20260927 23:28:21.374843 4184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19a88ff0709d4266b933cc526577127f [term 3 FOLLOWER]: Advancing to term 4
W20260927 23:28:21.385440 2380 catalog_manager.cc:4584] TS 5ce671f028a34ea4b8f0520c2689c9a1 (127.2.57.1:35657): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bbeeccc19a504e7984b293127766daf0: Network error: Client connection negotiation failed: client connection to 127.2.57.1:35657: connect: Connection refused (error 111)
I20260927 23:28:21.385756 3072 tablet_bootstrap.cc:492] T a8df6d0652634def9c8946c346dea702 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:21.386096 3072 ts_tablet_manager.cc:1397] T a8df6d0652634def9c8946c346dea702 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.100s user 0.009s sys 0.006s
I20260927 23:28:21.388243 3072 raft_consensus.cc:348] T a8df6d0652634def9c8946c346dea702 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.388835 3072 raft_consensus.cc:374] T a8df6d0652634def9c8946c346dea702 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:21.389050 3072 raft_consensus.cc:729] T a8df6d0652634def9c8946c346dea702 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:21.389601 3072 consensus_queue.cc:260] T a8df6d0652634def9c8946c346dea702 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.409111 3072 ts_tablet_manager.cc:1428] T a8df6d0652634def9c8946c346dea702 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.023s user 0.005s sys 0.001s
I20260927 23:28:21.401273 4197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:28:21.430198 3072 tablet_bootstrap.cc:492] T 1542b6be05634f6a88694df446373fe9 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
I20260927 23:28:21.436506 3072 tablet_bootstrap.cc:654] T 1542b6be05634f6a88694df446373fe9 P de216a4a8e4b4e1b857d8647a3b05f6d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:21.459878 3072 tablet_bootstrap.cc:492] T 1542b6be05634f6a88694df446373fe9 P de216a4a8e4b4e1b857d8647a3b05f6d: No bootstrap required, opened a new log
I20260927 23:28:21.460253 3072 ts_tablet_manager.cc:1397] T 1542b6be05634f6a88694df446373fe9 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent bootstrapping tablet: real 0.030s user 0.010s sys 0.003s
I20260927 23:28:21.462661 3072 raft_consensus.cc:348] T 1542b6be05634f6a88694df446373fe9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.463282 3072 raft_consensus.cc:374] T 1542b6be05634f6a88694df446373fe9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:21.463518 3072 raft_consensus.cc:729] T 1542b6be05634f6a88694df446373fe9 P de216a4a8e4b4e1b857d8647a3b05f6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de216a4a8e4b4e1b857d8647a3b05f6d, State: Initialized, Role: FOLLOWER
I20260927 23:28:21.464061 3072 consensus_queue.cc:260] T 1542b6be05634f6a88694df446373fe9 P de216a4a8e4b4e1b857d8647a3b05f6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abe182b2afc64d908f692ef45eae1d51" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 41573 } } peers { permanent_uuid: "5ce671f028a34ea4b8f0520c2689c9a1" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 35657 } } peers { permanent_uuid: "de216a4a8e4b4e1b857d8647a3b05f6d" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 33209 } }
I20260927 23:28:21.488044 3072 ts_tablet_manager.cc:1428] T 1542b6be05634f6a88694df446373fe9 P de216a4a8e4b4e1b857d8647a3b05f6d: Time spent starting tablet: real 0.027s user 0.004s sys 0.000s
I20260927 23:28:21.534430 3072 tablet_bootstrap.cc:492] T 1a590434f92645cf9dffedef3e01bd65 P de216a4a8e4b4e1b857d8647a3b05f6d: Bootstrap starting.
W20260927 23:28:21.536906 3706 connection.cc:535] server connection from 127.2.57.2:34611 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 23:28:21.541778 2276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu with pid 2883
I20260927 23:28:21.554777 4197 catalog_manager.cc:669] Loaded metadata for table default.Table_1322a4efff174b9e8c09d2be9f8e4570 [id=01acb3e29c634be0bed0ecb718c10591]
I20260927 23:28:21.577701 4197 catalog_manager.cc:669] Loaded metadata for table default.Table_d31a1e740a2b42bd8569372845897579 [id=0240b7478a2140c0b03cbe472b1e8c27]
I20260927 23:28:21.589690 4197 catalog_manager.cc:669] Loaded metadata for table default.Table_8823e2b699aa4e13badf61047486415f [id=0292eee5ce5f45ebac4f8a0d817809bd]
I20260927 23:28:21.592002 4197 catalog_manager.cc:669] Loaded metadata for table default.Table_0df33121d2eb4445ac8b8c6a9ee44d24 [id=0a5810627a964e7caf66c69e7c3ec9b4]
I20260927 23:28:21.605074 4197 catalog_manager.cc:669] Loaded metadata for table default.Table_9e2be4d6dccd46449e29c796e22c2011 [id=0c99a649b8164ec98b568d9ce89e27f6]
I20260927 23:28:21.618767 4197 catalog_manager.cc:669] Loaded metadata for table default.Table_e10895f6ce7143a4b328dfe1818e5c64 [id=13d7d633f0d04401855679055a1ac024]
I20260927 23:28:21.620263 4197 catalog_manager.cc:669] Loaded metadata for table default.Table_9b16095aa7824f10a46040ac08836015 [id=157e4b5315e047478ba67e506794f77a]
W20260927 23:28:21.634459 2302 connection.cc:535] server connection from 127.2.57.3:44601 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 23:28:21.641919 4197 catalog_manager.cc:669] Loaded metadata for table default.Table_73f1344829a046f29f209adc7eceaf42 [id=16e24fbc3d434613aeead047b9d624ba]
I20260927 23:28:21.644122 2276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu with pid 2285
I20260927 23:28:21.712422 2276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu with pid 2355
I20260927 23:28:21.774817 2276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu with pid 3298
2026-09-27T23:28:21Z chronyd exiting
I20260927 23:28:21.823202 2276 master-stress-test.cc:586] Tables created: 167
I20260927 23:28:21.823449 2276 master-stress-test.cc:587] Tables altered: 72
I20260927 23:28:21.823596 2276 master-stress-test.cc:588] Tables deleted: 56
I20260927 23:28:21.823736 2276 master-stress-test.cc:589] Tablets replaced: 25
I20260927 23:28:21.823864 2276 master-stress-test.cc:590] Masters restarted: 1
[ OK ] CatalogManagerConfigurations/MasterStressTest.Test/0 (40834 ms)
[ RUN ] CatalogManagerConfigurations/MasterStressTest.Test/3
I20260927 23:28:21.898428 2276 test_util.cc:268] Using random seed: 1311032484
2026-09-27T23:28:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T23:28:21Z Disabled control of system clock
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task6z6gmC/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task6z6gmC/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL: jdbc:derby:/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/metadb;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.derby.sql
Initialization script completed
schemaTool completed
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/security.properties
2026-09-27 23:28:29: Starting Hive Metastore Server
Picked up JAVA_TOOL_OPTIONS: -Djava.net.preferIPv4Stack=true -Dderby.locks.deadlockTimeout=1 -Dderby.stream.error.file=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/derby.log -Djava.security.properties=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/security.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/tmp/dist-test-task6z6gmC/thirdparty/src/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/tmp/dist-test-task6z6gmC/thirdparty/src/hadoop-3.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Starting hive metastore on port 0
I20260927 23:28:38.160720 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.62:35155
--webserver_interface=127.2.57.62
--webserver_port=0
--builtin_ntp_servers=127.2.57.20:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--hive_metastore_uris=thrift://127.0.0.1:36031
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260927 23:28:38.493216 4444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:28:38.493896 4444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:28:38.494421 4444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:28:38.528601 4444 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260927 23:28:38.528920 4444 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260927 23:28:38.529132 4444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:28:38.529341 4444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:28:38.529541 4444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:28:38.529736 4444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:28:38.568871 4444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:36031
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.62:35155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.57.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:28:38.570387 4444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:28:38.572273 4444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:28:38.593366 4451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:28:38.593439 4450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:38.596010 4444 server_base.cc:1034] running on GCE node
W20260927 23:28:38.595018 4453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:39.709949 4444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:28:39.713492 4444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:28:39.715008 4444 hybrid_clock.cc:648] HybridClock initialized: now 1790551719714959 us; error 63 us; skew 500 ppm
I20260927 23:28:39.715931 4444 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:28:39.728765 4444 webserver.cc:460] Webserver started at http://127.2.57.62:34737/ using document root <none> and password file <none>
I20260927 23:28:39.729858 4444 fs_manager.cc:359] Metadata directory not provided
I20260927 23:28:39.730091 4444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:28:39.730587 4444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:28:39.736130 4444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data/instance:
uuid: "d9ed71d97661411891f2d2de4cdc2269"
format_stamp: "Formatted at 2026-09-27 23:28:39 on dist-test-slave-jbx2"
I20260927 23:28:39.737421 4444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/wal/instance:
uuid: "d9ed71d97661411891f2d2de4cdc2269"
format_stamp: "Formatted at 2026-09-27 23:28:39 on dist-test-slave-jbx2"
I20260927 23:28:39.745810 4444 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260927 23:28:39.752249 4460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:28:39.753655 4444 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260927 23:28:39.754029 4444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/wal
uuid: "d9ed71d97661411891f2d2de4cdc2269"
format_stamp: "Formatted at 2026-09-27 23:28:39 on dist-test-slave-jbx2"
I20260927 23:28:39.754420 4444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:28:39.797353 4444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:28:39.799054 4444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:28:39.799543 4444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:28:39.882607 4444 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.62:35155
I20260927 23:28:39.882712 4511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.62:35155 every 8 connection(s)
I20260927 23:28:39.885684 4444 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data/info.pb
I20260927 23:28:39.889816 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 4444
I20260927 23:28:39.890256 2276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/wal/instance
I20260927 23:28:39.891687 4512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:39.893075 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.61:46527
--webserver_interface=127.2.57.61
--webserver_port=0
--builtin_ntp_servers=127.2.57.20:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--hive_metastore_uris=thrift://127.0.0.1:36031
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260927 23:28:39.907174 4512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:28:39.940104 4512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } has no permanent_uuid. Determining permanent_uuid...
W20260927 23:28:39.943285 4462 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:28:39.947585 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:46527: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:28:39.997246 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } attempt: 1
W20260927 23:28:40.001875 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:46527: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:28:40.070322 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } attempt: 2
W20260927 23:28:40.074332 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:46527: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:28:40.165881 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } attempt: 3
W20260927 23:28:40.170100 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:46527: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:28:40.237021 4513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:28:40.237733 4513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:28:40.238181 4513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:28:40.272836 4513 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260927 23:28:40.273170 4513 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260927 23:28:40.273443 4513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:28:40.273669 4513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:28:40.273916 4513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:28:40.274169 4513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:28:40.312942 4513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/wal
--hive_metastore_uris=thrift://127.0.0.1:36031
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.61:46527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.57.61
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:28:40.313668 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } attempt: 4
I20260927 23:28:40.314483 4513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:28:40.316861 4513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:28:40.317929 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:46527: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:28:40.337827 4525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:40.617383 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } attempt: 5
W20260927 23:28:40.621395 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:46527: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:28:41.168983 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } attempt: 6
W20260927 23:28:41.176798 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.61:46527: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:28:40.337842 4522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:28:40.337854 4523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:40.339483 4513 server_base.cc:1034] running on GCE node
I20260927 23:28:41.461179 4513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:28:41.464124 4513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:28:41.465545 4513 hybrid_clock.cc:648] HybridClock initialized: now 1790551721465521 us; error 48 us; skew 500 ppm
I20260927 23:28:41.466441 4513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:28:41.473658 4513 webserver.cc:460] Webserver started at http://127.2.57.61:38499/ using document root <none> and password file <none>
I20260927 23:28:41.474807 4513 fs_manager.cc:359] Metadata directory not provided
I20260927 23:28:41.475026 4513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:28:41.475493 4513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:28:41.481323 4513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/data/instance:
uuid: "f8460f73b40c41a28ec020fc04203564"
format_stamp: "Formatted at 2026-09-27 23:28:41 on dist-test-slave-jbx2"
I20260927 23:28:41.482645 4513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/wal/instance:
uuid: "f8460f73b40c41a28ec020fc04203564"
format_stamp: "Formatted at 2026-09-27 23:28:41 on dist-test-slave-jbx2"
I20260927 23:28:41.491007 4513 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.001s sys 0.008s
I20260927 23:28:41.497572 4534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:28:41.498860 4513 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260927 23:28:41.499218 4513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/wal
uuid: "f8460f73b40c41a28ec020fc04203564"
format_stamp: "Formatted at 2026-09-27 23:28:41 on dist-test-slave-jbx2"
I20260927 23:28:41.499583 4513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:28:41.538198 4513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:28:41.539975 4513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:28:41.540531 4513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:28:41.622252 4513 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.61:46527
I20260927 23:28:41.622334 4586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.61:46527 every 8 connection(s)
I20260927 23:28:41.625425 4513 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/data/info.pb
I20260927 23:28:41.631455 4587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:41.633891 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 4513
I20260927 23:28:41.634516 2276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-1/wal/instance
I20260927 23:28:41.637460 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.60:33749
--webserver_interface=127.2.57.60
--webserver_port=0
--builtin_ntp_servers=127.2.57.20:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--hive_metastore_uris=thrift://127.0.0.1:36031
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
I20260927 23:28:41.644742 4587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:28:41.678434 4587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:28:41.696678 4587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } has no permanent_uuid. Determining permanent_uuid...
W20260927 23:28:41.699268 4538 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
W20260927 23:28:41.704247 4587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
I20260927 23:28:41.753940 4587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 1
W20260927 23:28:41.758117 4587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
I20260927 23:28:41.826581 4587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 2
W20260927 23:28:41.830705 4587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
I20260927 23:28:41.922102 4587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 3
W20260927 23:28:41.926386 4587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
W20260927 23:28:41.989504 4588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:28:41.990240 4588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:28:41.990756 4588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:28:42.025362 4588 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260927 23:28:42.025703 4588 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260927 23:28:42.025959 4588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:28:42.026201 4588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:28:42.026446 4588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:28:42.026696 4588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:28:42.065708 4588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/wal
--hive_metastore_uris=thrift://127.0.0.1:36031
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.60:33749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.57.60
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:28:42.067234 4588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:28:42.069144 4588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 23:28:42.069823 4587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 4
W20260927 23:28:42.073979 4587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
W20260927 23:28:42.090078 4601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:42.237427 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } attempt: 7
I20260927 23:28:42.251840 4512 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } has no permanent_uuid. Determining permanent_uuid...
W20260927 23:28:42.258176 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
I20260927 23:28:42.316699 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 1
W20260927 23:28:42.321338 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
I20260927 23:28:42.373483 4587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 5
W20260927 23:28:42.378080 4587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
I20260927 23:28:42.402788 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 2
W20260927 23:28:42.407292 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
I20260927 23:28:42.492733 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 3
W20260927 23:28:42.497143 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
I20260927 23:28:42.637629 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 4
W20260927 23:28:42.641548 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
I20260927 23:28:42.925004 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 5
I20260927 23:28:42.925537 4587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 6
W20260927 23:28:42.929162 4512 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
W20260927 23:28:42.933470 4587 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.57.60:33749: Network error: Client connection negotiation failed: client connection to 127.2.57.60:33749: connect: Connection refused (error 111)
W20260927 23:28:42.090150 4599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:28:42.090188 4598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:42.091843 4588 server_base.cc:1034] running on GCE node
I20260927 23:28:43.224943 4588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:28:43.228442 4588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:28:43.229924 4588 hybrid_clock.cc:648] HybridClock initialized: now 1790551723229847 us; error 108 us; skew 500 ppm
I20260927 23:28:43.230865 4588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:28:43.238564 4588 webserver.cc:460] Webserver started at http://127.2.57.60:35811/ using document root <none> and password file <none>
I20260927 23:28:43.239688 4588 fs_manager.cc:359] Metadata directory not provided
I20260927 23:28:43.239914 4588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:28:43.240460 4588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:28:43.246156 4588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/data/instance:
uuid: "83cd22a89b7846999c0e4dfab80ebe9d"
format_stamp: "Formatted at 2026-09-27 23:28:43 on dist-test-slave-jbx2"
I20260927 23:28:43.247485 4588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/wal/instance:
uuid: "83cd22a89b7846999c0e4dfab80ebe9d"
format_stamp: "Formatted at 2026-09-27 23:28:43 on dist-test-slave-jbx2"
I20260927 23:28:43.256146 4588 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.004s
I20260927 23:28:43.263051 4613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:28:43.264458 4588 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.002s
I20260927 23:28:43.264855 4588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/wal
uuid: "83cd22a89b7846999c0e4dfab80ebe9d"
format_stamp: "Formatted at 2026-09-27 23:28:43 on dist-test-slave-jbx2"
I20260927 23:28:43.265232 4588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:28:43.302973 4588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:28:43.304713 4588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:28:43.305193 4588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:28:43.386925 4588 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.60:33749
I20260927 23:28:43.387020 4664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.60:33749 every 8 connection(s)
I20260927 23:28:43.390030 4588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/data/info.pb
I20260927 23:28:43.393918 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 4588
I20260927 23:28:43.394383 2276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-2/wal/instance
I20260927 23:28:43.395845 4665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:43.416239 4665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:28:43.443146 4665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:28:43.455242 4665 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } has no permanent_uuid. Determining permanent_uuid...
I20260927 23:28:43.474300 4665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Bootstrap starting.
I20260927 23:28:43.481487 4665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:43.481632 4512 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 6
I20260927 23:28:43.484280 4665 log.cc:824] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Log is configured to *not* fsync() on all Append() calls
I20260927 23:28:43.490547 4665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: No bootstrap required, opened a new log
I20260927 23:28:43.504043 4512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: Bootstrap starting.
I20260927 23:28:43.511140 4512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:43.513737 4512 log.cc:824] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: Log is configured to *not* fsync() on all Append() calls
I20260927 23:28:43.514886 4665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:43.515599 4665 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:43.515869 4665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83cd22a89b7846999c0e4dfab80ebe9d, State: Initialized, Role: FOLLOWER
I20260927 23:28:43.516803 4665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:43.519495 4512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: No bootstrap required, opened a new log
I20260927 23:28:43.519867 4674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:28:43.520821 4674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:28:43.521410 4665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:28:43.550460 4512 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:43.551826 4512 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:43.552246 4512 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9ed71d97661411891f2d2de4cdc2269, State: Initialized, Role: FOLLOWER
I20260927 23:28:43.553699 4512 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
W20260927 23:28:43.561064 4619 tablet.cc:2369] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:28:43.562506 4689 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:28:43.563585 4689 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:28:43.568234 4512 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: configured and running, proceeding with master startup.
W20260927 23:28:43.568367 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.568892 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.587792 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.588351 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.556798 4588 master_runner.cc:186] Error getting master registration for 127.2.57.62:35155: OK, instance_id { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" instance_seqno: 1790551719859402 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 23:28:43.595046 4588 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260927 23:28:43.608443 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.608834 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.625070 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.625568 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.640170 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.640682 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.655334 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.655807 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.671069 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.675760 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.692122 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.697654 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.703382 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.703899 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.712270 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.712762 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.724630 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.725008 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.727110 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.727512 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.739562 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.739996 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.743314 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.752873 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.754819 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.755336 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.760840 4674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:43.761464 4674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:43.765570 4487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269" is_pre_election: true
I20260927 23:28:43.766593 4487 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83cd22a89b7846999c0e4dfab80ebe9d in term 0.
I20260927 23:28:43.768517 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d, d9ed71d97661411891f2d2de4cdc2269; no voters:
W20260927 23:28:43.770030 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.770426 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.773294 4674 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:28:43.772185 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.776440 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.773469 4708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:43.780746 4708 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:43.781108 4708 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:43.785020 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.785368 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.789506 4708 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
W20260927 23:28:43.800043 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.800429 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.693549 4444 master_runner.cc:186] Error getting master registration for 127.2.57.61:46527: OK, instance_id { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" instance_seqno: 1790551721600446 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 23:28:43.804396 4444 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 23:28:43.806942 4708 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 1 election: Requested vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
I20260927 23:28:43.808871 4487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269"
I20260927 23:28:43.809577 4487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:43.810038 4466 tablet.cc:2369] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 23:28:43.822996 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.824429 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.824920 4487 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83cd22a89b7846999c0e4dfab80ebe9d in term 1.
I20260927 23:28:43.826398 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d, d9ed71d97661411891f2d2de4cdc2269; no voters:
I20260927 23:28:43.827466 4708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:43.828254 4708 raft_consensus.cc:686] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 1 LEADER]: Becoming Leader. State: Replica: 83cd22a89b7846999c0e4dfab80ebe9d, State: Running, Role: LEADER
I20260927 23:28:43.829428 4708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
W20260927 23:28:43.841958 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.842384 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.849253 4674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83cd22a89b7846999c0e4dfab80ebe9d. Latest consensus state: current_term: 1 leader_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:28:43.850157 4674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: This master's current role is: LEADER
W20260927 23:28:43.854211 4688 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.855832 4688 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260927 23:28:43.857470 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.857842 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.864359 4710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:28:43.872707 4710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260927 23:28:43.875229 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.876029 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.902383 4487 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Refusing update from remote peer 83cd22a89b7846999c0e4dfab80ebe9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:28:43.904479 4674 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:28:43.922695 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.924891 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.942441 4689 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83cd22a89b7846999c0e4dfab80ebe9d. Latest consensus state: current_term: 1 leader_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:28:43.943290 4689 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:28:43.951452 4711 mvcc.cc:205] Tried to move back new op lower bound from 7334099861075996672 to 7334099860824797184. Current Snapshot: MvccSnapshot[applied={T|T < 7334099861075996672}]
I20260927 23:28:43.955878 4712 mvcc.cc:205] Tried to move back new op lower bound from 7334099861075996672 to 7334099860824797184. Current Snapshot: MvccSnapshot[applied={T|T < 7334099861075996672}]
I20260927 23:28:43.957397 4708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:28:43.958209 4708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: This master's current role is: LEADER
W20260927 23:28:43.957703 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260927 23:28:43.960186 4710 catalog_manager.cc:1347] Generated new cluster ID: 6dc11a7b873d4b11830ae46041eb6de8
I20260927 23:28:43.960492 4710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260927 23:28:43.960811 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.968564 4689 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:28:43.969403 4689 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: This master's current role is: FOLLOWER
W20260927 23:28:43.987316 4706 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 23:28:43.987704 4706 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 23:28:43.993861 4587 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } attempt: 7
I20260927 23:28:44.001976 4710 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 23:28:44.008446 4706 catalog_manager.cc:1259] Loaded cluster ID: 6dc11a7b873d4b11830ae46041eb6de8
I20260927 23:28:44.008731 4706 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: success
I20260927 23:28:44.016710 4710 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:28:44.029820 4706 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: acquiring CA information for follower catalog manager: success
W20260927 23:28:44.035578 4706 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260927 23:28:44.056982 4710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Generated new TSK 0
I20260927 23:28:44.061985 4710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:28:44.063884 4587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: Bootstrap starting.
I20260927 23:28:44.067658 4706 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 23:28:44.076051 4710 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260927 23:28:44.076112 4587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:44.089247 4710 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: -1
I20260927 23:28:44.092980 4587 log.cc:824] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: Log is configured to *not* fsync() on all Append() calls
I20260927 23:28:44.098344 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.57.1:0
--local_ip_for_outbound_sockets=127.2.57.1
--webserver_interface=127.2.57.1
--webserver_port=0
--tserver_master_addrs=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--builtin_ntp_servers=127.2.57.20:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260927 23:28:44.118242 4587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: No bootstrap required, opened a new log
I20260927 23:28:44.188529 4587 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:44.189700 4587 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:44.190068 4587 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8460f73b40c41a28ec020fc04203564, State: Initialized, Role: FOLLOWER
I20260927 23:28:44.191279 4587 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:44.196560 4720 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:28:44.199105 4720 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:28:44.201809 4587 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 23:28:44.202649 4560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:44.208685 4562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:28:44.211958 4561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564"
W20260927 23:28:44.213984 4561 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53516 (request call id 2) took 405 ms (client timeout 532 ms). Trace:
W20260927 23:28:44.214397 4561 rpcz_store.cc:269] 0927 23:28:43.808400 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:43.810306 (+ 1906us) service_pool.cc:227] Handling call
0927 23:28:44.213889 (+403583us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260927 23:28:44.221565 4560 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Refusing update from remote peer 83cd22a89b7846999c0e4dfab80ebe9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:28:44.225059 4674 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:44.310811 4720 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83cd22a89b7846999c0e4dfab80ebe9d. Latest consensus state: current_term: 1 leader_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:28:44.313573 4720 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:28:44.312537 4728 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:28:44.314463 4728 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:28:44.434043 4740 catalog_manager.cc:1259] Loaded cluster ID: 6dc11a7b873d4b11830ae46041eb6de8
I20260927 23:28:44.434365 4740 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: loading cluster ID for follower catalog manager: success
I20260927 23:28:44.443280 4740 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: acquiring CA information for follower catalog manager: success
I20260927 23:28:44.462352 4740 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 23:28:44.586161 4742 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:28:44.586864 4742 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:44.590481 4561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:28:44.594470 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:28:44.597527 4464 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, f8460f73b40c41a28ec020fc04203564
I20260927 23:28:44.600723 4742 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:28:44.601876 4742 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:44.732371 4728 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:28:44.732940 4728 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:44.735387 4728 leader_election.cc:288] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:28:44.738055 4486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269" is_pre_election: true
I20260927 23:28:44.739372 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:28:44.741828 4537 leader_election.cc:302] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8460f73b40c41a28ec020fc04203564; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, d9ed71d97661411891f2d2de4cdc2269
I20260927 23:28:44.742781 4728 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:44.876096 4748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:28:44.884019 4748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: This master's current role is: LEADER
W20260927 23:28:44.948769 4719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:28:44.949522 4719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:28:44.950527 4719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:28:45.017045 4719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:28:45.018975 4719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.57.1
I20260927 23:28:45.092742 4719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.57.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.57.1
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.57.1
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:28:45.094733 4719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:28:45.099371 4719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 23:28:45.122362 4743 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:28:45.122843 4743 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:45.125654 4561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:28:45.127553 4743 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:28:45.129904 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:28:45.131786 4464 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, f8460f73b40c41a28ec020fc04203564
I20260927 23:28:45.132977 4743 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:28:45.164903 4753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:46.230897 4758 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:28:46.232869 4758 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:46.248500 4486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269" is_pre_election: true
I20260927 23:28:46.251919 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:28:46.253675 4538 leader_election.cc:302] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8460f73b40c41a28ec020fc04203564; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, d9ed71d97661411891f2d2de4cdc2269
I20260927 23:28:46.259948 4758 leader_election.cc:288] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:28:46.261127 4758 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:45.174727 4719 server_base.cc:1034] running on GCE node
W20260927 23:28:45.176800 4756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:28:45.174571 4752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:46.788125 4719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:28:46.799741 4760 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:28:46.800225 4760 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:46.801620 4719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:28:46.803108 4719 hybrid_clock.cc:648] HybridClock initialized: now 1790551726803021 us; error 107 us; skew 500 ppm
I20260927 23:28:46.804237 4719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:28:46.806088 4486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269" is_pre_election: true
I20260927 23:28:46.807789 4760 leader_election.cc:288] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:28:46.808902 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:28:46.810401 4538 leader_election.cc:302] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8460f73b40c41a28ec020fc04203564; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, d9ed71d97661411891f2d2de4cdc2269
I20260927 23:28:46.811146 4760 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:46.816529 4719 webserver.cc:460] Webserver started at http://127.2.57.1:34843/ using document root <none> and password file <none>
I20260927 23:28:46.818023 4719 fs_manager.cc:359] Metadata directory not provided
I20260927 23:28:46.818405 4719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:28:46.819155 4719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:28:46.826682 4719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/data/instance:
uuid: "7e62b4926a8448b2b5f4923553328bca"
format_stamp: "Formatted at 2026-09-27 23:28:46 on dist-test-slave-jbx2"
I20260927 23:28:46.828431 4719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/instance:
uuid: "7e62b4926a8448b2b5f4923553328bca"
format_stamp: "Formatted at 2026-09-27 23:28:46 on dist-test-slave-jbx2"
I20260927 23:28:46.839783 4719 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.008s sys 0.000s
I20260927 23:28:46.848115 4765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:28:46.849584 4719 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260927 23:28:46.849989 4719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal
uuid: "7e62b4926a8448b2b5f4923553328bca"
format_stamp: "Formatted at 2026-09-27 23:28:46 on dist-test-slave-jbx2"
I20260927 23:28:46.850441 4719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:28:46.929924 4719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:28:46.931942 4719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:28:46.932487 4719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:28:46.936949 4719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:28:46.943046 4719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:28:46.943351 4719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:28:46.943640 4719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:28:46.943835 4719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:28:47.142344 4719 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.1:40571
I20260927 23:28:47.142431 4877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.1:40571 every 8 connection(s)
I20260927 23:28:47.145820 4719 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/data/info.pb
I20260927 23:28:47.153231 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 4719
I20260927 23:28:47.153733 2276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/instance
I20260927 23:28:47.182173 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.57.2:0
--local_ip_for_outbound_sockets=127.2.57.2
--webserver_interface=127.2.57.2
--webserver_port=0
--tserver_master_addrs=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--builtin_ntp_servers=127.2.57.20:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260927 23:28:47.196702 4878 heartbeater.cc:347] Connected to a master server at 127.2.57.60:33749
I20260927 23:28:47.197312 4878 heartbeater.cc:464] Registering TS with master...
I20260927 23:28:47.198606 4878 heartbeater.cc:511] Master 127.2.57.60:33749 requested a full tablet report, sending...
I20260927 23:28:47.202637 4630 ts_manager.cc:194] Registered new tserver with Master: 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
I20260927 23:28:47.205945 4630 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.57.1:36749
I20260927 23:28:47.227650 4879 heartbeater.cc:347] Connected to a master server at 127.2.57.61:46527
I20260927 23:28:47.228065 4879 heartbeater.cc:464] Registering TS with master...
I20260927 23:28:47.228814 4879 heartbeater.cc:511] Master 127.2.57.61:46527 requested a full tablet report, sending...
I20260927 23:28:47.231978 4552 ts_manager.cc:194] Registered new tserver with Master: 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
I20260927 23:28:47.235538 4880 heartbeater.cc:347] Connected to a master server at 127.2.57.62:35155
I20260927 23:28:47.235915 4880 heartbeater.cc:464] Registering TS with master...
I20260927 23:28:47.236558 4880 heartbeater.cc:511] Master 127.2.57.62:35155 requested a full tablet report, sending...
I20260927 23:28:47.238965 4477 ts_manager.cc:194] Registered new tserver with Master: 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
I20260927 23:28:47.387661 4888 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:28:47.388164 4888 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:47.390105 4888 leader_election.cc:288] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:28:47.391032 4486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269" is_pre_election: true
I20260927 23:28:47.392349 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:28:47.393810 4538 leader_election.cc:302] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8460f73b40c41a28ec020fc04203564; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, d9ed71d97661411891f2d2de4cdc2269
I20260927 23:28:47.394557 4888 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:28:47.554354 4885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:28:47.555020 4885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:28:47.555685 4885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:28:47.589983 4885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:28:47.591029 4885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.57.2
I20260927 23:28:47.632905 4885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.57.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.57.2
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.57.2
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:28:47.634573 4885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:28:47.636626 4885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:28:47.659332 4895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:47.715399 4878 heartbeater.cc:503] Master 127.2.57.60:33749 was elected leader, sending a full tablet report...
I20260927 23:28:48.518424 4901 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:28:48.519145 4901 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:48.521435 4901 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:28:48.522858 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:28:48.522926 4561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:28:48.525154 4462 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, f8460f73b40c41a28ec020fc04203564
I20260927 23:28:48.526386 4901 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:28:47.659372 4894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:28:47.660481 4897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:47.662350 4885 server_base.cc:1034] running on GCE node
I20260927 23:28:48.810344 4885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:28:48.813362 4885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:28:48.814972 4885 hybrid_clock.cc:648] HybridClock initialized: now 1790551728814923 us; error 53 us; skew 500 ppm
I20260927 23:28:48.816143 4885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:28:48.824707 4885 webserver.cc:460] Webserver started at http://127.2.57.2:40395/ using document root <none> and password file <none>
I20260927 23:28:48.825762 4885 fs_manager.cc:359] Metadata directory not provided
I20260927 23:28:48.825968 4885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:28:48.826480 4885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:28:48.832062 4885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/data/instance:
uuid: "bad92a10de2c4459827cdbb18854d76d"
format_stamp: "Formatted at 2026-09-27 23:28:48 on dist-test-slave-jbx2"
I20260927 23:28:48.833408 4885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/instance:
uuid: "bad92a10de2c4459827cdbb18854d76d"
format_stamp: "Formatted at 2026-09-27 23:28:48 on dist-test-slave-jbx2"
I20260927 23:28:48.841801 4885 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260927 23:28:48.848179 4906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:28:48.849468 4885 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260927 23:28:48.849835 4885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal
uuid: "bad92a10de2c4459827cdbb18854d76d"
format_stamp: "Formatted at 2026-09-27 23:28:48 on dist-test-slave-jbx2"
I20260927 23:28:48.850314 4885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:28:48.886828 4885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:28:48.889081 4885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:28:48.889711 4885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:28:48.893555 4885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:28:48.900233 4885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:28:48.900581 4885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260927 23:28:48.900887 4885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:28:48.901098 4885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:28:49.072249 4885 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.2:45717
I20260927 23:28:49.072396 5018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.2:45717 every 8 connection(s)
I20260927 23:28:49.075415 4885 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/data/info.pb
I20260927 23:28:49.083026 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 4885
I20260927 23:28:49.083648 2276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/instance
I20260927 23:28:49.102038 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.57.3:0
--local_ip_for_outbound_sockets=127.2.57.3
--webserver_interface=127.2.57.3
--webserver_port=0
--tserver_master_addrs=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--builtin_ntp_servers=127.2.57.20:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--heartbeat_rpc_timeout_ms=1000
--heartbeat_max_failures_before_backoff=1
--heartbeat_interval_ms=500 with env {}
I20260927 23:28:49.142797 5030 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:28:49.143388 5030 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:49.152753 5021 heartbeater.cc:347] Connected to a master server at 127.2.57.62:35155
I20260927 23:28:49.153358 5021 heartbeater.cc:464] Registering TS with master...
I20260927 23:28:49.154585 4561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:28:49.154874 5021 heartbeater.cc:511] Master 127.2.57.62:35155 requested a full tablet report, sending...
I20260927 23:28:49.157965 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:28:49.160231 5020 heartbeater.cc:347] Connected to a master server at 127.2.57.61:46527
I20260927 23:28:49.160163 4477 ts_manager.cc:194] Registered new tserver with Master: bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:49.160696 5020 heartbeater.cc:464] Registering TS with master...
I20260927 23:28:49.160467 4464 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, f8460f73b40c41a28ec020fc04203564
I20260927 23:28:49.161762 5020 heartbeater.cc:511] Master 127.2.57.61:46527 requested a full tablet report, sending...
I20260927 23:28:49.164098 5030 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:28:49.164861 5030 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:49.165791 4552 ts_manager.cc:194] Registered new tserver with Master: bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:49.169530 5019 heartbeater.cc:347] Connected to a master server at 127.2.57.60:33749
I20260927 23:28:49.169945 5019 heartbeater.cc:464] Registering TS with master...
I20260927 23:28:49.170787 5019 heartbeater.cc:511] Master 127.2.57.60:33749 requested a full tablet report, sending...
I20260927 23:28:49.172827 4630 ts_manager.cc:194] Registered new tserver with Master: bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:49.174948 4630 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.57.2:52995
W20260927 23:28:49.489514 5029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 23:28:49.490077 5029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 23:28:49.490671 5029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 23:28:49.526427 5029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:28:49.527539 5029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.57.3
I20260927 23:28:49.567889 5029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--raft_heartbeat_interval_ms=1000
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.57.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.57.3
--webserver_port=0
--heartbeat_interval_ms=500
--heartbeat_max_failures_before_backoff=1
--heartbeat_rpc_timeout_ms=1000
--tserver_master_addrs=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.57.3
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:28:49.569622 5029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:28:49.571660 5029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 23:28:49.594022 5038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:49.679522 5019 heartbeater.cc:503] Master 127.2.57.60:33749 was elected leader, sending a full tablet report...
I20260927 23:28:49.697167 5043 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:28:49.697887 5043 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:49.699829 5043 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:28:49.700819 4561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:28:49.701288 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:28:49.702791 4464 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, f8460f73b40c41a28ec020fc04203564
I20260927 23:28:49.703507 5043 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:50.171906 5044 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:28:50.172513 5044 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:50.174780 5044 leader_election.cc:288] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:28:50.178222 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:28:50.178182 4486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269" is_pre_election: true
I20260927 23:28:50.180049 4537 leader_election.cc:302] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8460f73b40c41a28ec020fc04203564; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, d9ed71d97661411891f2d2de4cdc2269
I20260927 23:28:50.181126 5044 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:28:49.594072 5037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:28:49.595217 5040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:28:50.775424 5039 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Connection time-out
I20260927 23:28:50.775993 5029 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20260927 23:28:50.779685 5029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:28:50.782240 5029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 23:28:50.783677 5029 hybrid_clock.cc:648] HybridClock initialized: now 1790551730783634 us; error 72 us; skew 500 ppm
I20260927 23:28:50.784824 5029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260927 23:28:50.792716 5029 webserver.cc:460] Webserver started at http://127.2.57.3:41497/ using document root <none> and password file <none>
I20260927 23:28:50.793854 5029 fs_manager.cc:359] Metadata directory not provided
I20260927 23:28:50.794114 5029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 23:28:50.794656 5029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 23:28:50.800382 5029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/data/instance:
uuid: "f5ec98b2972d44838fa273eb7a10d01a"
format_stamp: "Formatted at 2026-09-27 23:28:50 on dist-test-slave-jbx2"
I20260927 23:28:50.801860 5029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal/instance:
uuid: "f5ec98b2972d44838fa273eb7a10d01a"
format_stamp: "Formatted at 2026-09-27 23:28:50 on dist-test-slave-jbx2"
I20260927 23:28:50.810631 5029 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.003s
I20260927 23:28:50.818001 5050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:28:50.819465 5029 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.003s
I20260927 23:28:50.819832 5029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal
uuid: "f5ec98b2972d44838fa273eb7a10d01a"
format_stamp: "Formatted at 2026-09-27 23:28:50 on dist-test-slave-jbx2"
I20260927 23:28:50.820192 5029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:28:50.870252 5029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:28:50.872087 5029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:28:50.872702 5029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:28:50.875891 5029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 23:28:50.880882 5029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 23:28:50.881213 5029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 23:28:50.881512 5029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 23:28:50.881742 5029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 23:28:51.049539 5029 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.3:45519
I20260927 23:28:51.049710 5162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.3:45519 every 8 connection(s)
I20260927 23:28:51.052557 5029 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/data/info.pb
I20260927 23:28:51.060879 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 5029
I20260927 23:28:51.061525 2276 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal/instance
I20260927 23:28:51.094527 5173 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:28:51.095185 5173 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:28:51.098556 5173 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:28:51.098771 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:28:51.098800 4561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:28:51.100742 4462 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, f8460f73b40c41a28ec020fc04203564
I20260927 23:28:51.101709 5173 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:28:51.134689 5164 heartbeater.cc:347] Connected to a master server at 127.2.57.61:46527
I20260927 23:28:51.135125 5165 heartbeater.cc:347] Connected to a master server at 127.2.57.62:35155
I20260927 23:28:51.135299 5164 heartbeater.cc:464] Registering TS with master...
I20260927 23:28:51.135725 5165 heartbeater.cc:464] Registering TS with master...
I20260927 23:28:51.136905 5165 heartbeater.cc:511] Master 127.2.57.62:35155 requested a full tablet report, sending...
I20260927 23:28:51.136924 5164 heartbeater.cc:511] Master 127.2.57.61:46527 requested a full tablet report, sending...
I20260927 23:28:51.138851 5163 heartbeater.cc:347] Connected to a master server at 127.2.57.60:33749
I20260927 23:28:51.140424 4477 ts_manager.cc:194] Registered new tserver with Master: f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:51.141569 5163 heartbeater.cc:464] Registering TS with master...
I20260927 23:28:51.142570 5163 heartbeater.cc:511] Master 127.2.57.60:33749 requested a full tablet report, sending...
I20260927 23:28:51.145529 4630 ts_manager.cc:194] Registered new tserver with Master: f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:51.146170 4552 ts_manager.cc:194] Registered new tserver with Master: f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:51.147827 4630 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.57.3:57013
I20260927 23:28:51.167069 2276 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 23:28:51.222811 4630 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_0275a380b3fe49989cf0c54688f42d05"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:51.225088 4629 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_d49f20a4a3204c20954377a8d7ac4b4b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:51.227638 4628 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_72baadd8af844c73bdd4465219a8601f"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:51.228322 4627 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_86bcd06c32004328b4e6830ef49b41c5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:28:51.241221 4629 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_d49f20a4a3204c20954377a8d7ac4b4b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:51.243185 4630 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_0275a380b3fe49989cf0c54688f42d05 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:51.279922 2276 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu with pid 4444
I20260927 23:28:51.653173 5163 heartbeater.cc:503] Master 127.2.57.60:33749 was elected leader, sending a full tablet report...
W20260927 23:28:51.665447 5165 heartbeater.cc:644] Failed to heartbeat to 127.2.57.62:35155 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:28:51.691921 5021 heartbeater.cc:644] Failed to heartbeat to 127.2.57.62:35155 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:28:51.773473 4630 catalog_manager.cc:2148] created HMS catalog entry for table [id=c2a82848ea2847feb1c711f38247865d]
W20260927 23:28:51.788646 4880 heartbeater.cc:644] Failed to heartbeat to 127.2.57.62:35155 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:28:51.812119 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:28:51.822300 2276 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
/tmp/dist-test-task6z6gmC/build/tsan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.62:35155
--webserver_interface=127.2.57.62
--webserver_port=34737
--builtin_ntp_servers=127.2.57.20:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--hive_metastore_uris=thrift://127.0.0.1:36031
--hive_metastore_notification_log_poll_period_seconds=1
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--catalog_manager_bg_task_wait_ms=10
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--leader_failure_max_missed_heartbeat_periods=1.0
--log_preallocate_segments=false with env {}
W20260927 23:28:51.845041 4616 consensus_peers.cc:487] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d -> Peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Couldn't send request to peer d9ed71d97661411891f2d2de4cdc2269. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:51.854861 4629 catalog_manager.cc:2148] created HMS catalog entry for table [id=ec636e7b20704b56bd3191822c076863]
I20260927 23:28:51.855283 4630 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "default.Table_0275a380b3fe49989cf0c54688f42d05" } new_table_name: "default.Table_e141ccac1ae746388643f3ee5f04cdc1" modify_external_catalogs: true
W20260927 23:28:51.865011 4628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_72baadd8af844c73bdd4465219a8601f in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:51.865656 4627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_86bcd06c32004328b4e6830ef49b41c5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:51.890136 4629 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "default.Table_d49f20a4a3204c20954377a8d7ac4b4b" } new_table_name: "default.Table_23bd70063a25487ab35a76f05dd4b81b" modify_external_catalogs: true
I20260927 23:28:51.950757 4813 tablet_service.cc:1467] Processing CreateTablet for tablet e4e5cb0d645846159e09ab71a897f6df (DEFAULT_TABLE table=default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:51.953320 4954 tablet_service.cc:1467] Processing CreateTablet for tablet e4e5cb0d645846159e09ab71a897f6df (DEFAULT_TABLE table=default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:51.951414 4953 tablet_service.cc:1467] Processing CreateTablet for tablet 29eb6018ba1e424ba628221fe56a3cc1 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:51.953145 4812 tablet_service.cc:1467] Processing CreateTablet for tablet 29eb6018ba1e424ba628221fe56a3cc1 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:51.955329 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29eb6018ba1e424ba628221fe56a3cc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:51.955431 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29eb6018ba1e424ba628221fe56a3cc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:51.961225 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4e5cb0d645846159e09ab71a897f6df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:51.965384 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4e5cb0d645846159e09ab71a897f6df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:51.985975 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 29eb6018ba1e424ba628221fe56a3cc1 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:51.989869 4628 catalog_manager.cc:2148] created HMS catalog entry for table [id=409d40c7e476447caaf9bb66f61e4b82]
I20260927 23:28:51.999625 5098 tablet_service.cc:1467] Processing CreateTablet for tablet e4e5cb0d645846159e09ab71a897f6df (DEFAULT_TABLE table=default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:51.988178 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29eb6018ba1e424ba628221fe56a3cc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.023360 5219 tablet_bootstrap.cc:492] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:52.025794 4628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "default.Table_72baadd8af844c73bdd4465219a8601f" } modify_external_catalogs: true
I20260927 23:28:52.027576 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4e5cb0d645846159e09ab71a897f6df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.026794 4628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "default.Table_72baadd8af844c73bdd4465219a8601f" } modify_external_catalogs: true
I20260927 23:28:52.031921 5219 tablet_bootstrap.cc:654] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:52.033439 4626 master_service.cc:946] ReplaceTablet: received request to replace tablet e4e5cb0d645846159e09ab71a897f6df from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:52.042053 4625 master_service.cc:946] ReplaceTablet: received request to replace tablet e4e5cb0d645846159e09ab71a897f6df from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:52.056617 4624 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_0ec8d1f3b75e4c44b6b60deb61245f76"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:52.062333 5219 log.cc:824] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca: Log is configured to *not* fsync() on all Append() calls
I20260927 23:28:52.063818 4813 tablet_service.cc:1467] Processing CreateTablet for tablet baae829e8b9c4e7f8c2d3a1414666fcb (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.065028 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baae829e8b9c4e7f8c2d3a1414666fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.064634 4952 tablet_service.cc:1467] Processing CreateTablet for tablet baae829e8b9c4e7f8c2d3a1414666fcb (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.065339 5096 tablet_service.cc:1467] Processing CreateTablet for tablet baae829e8b9c4e7f8c2d3a1414666fcb (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.066063 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baae829e8b9c4e7f8c2d3a1414666fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.066527 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet baae829e8b9c4e7f8c2d3a1414666fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.095559 5219 tablet_bootstrap.cc:492] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:52.096206 5219 ts_tablet_manager.cc:1397] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.074s user 0.012s sys 0.004s
I20260927 23:28:52.109272 4623 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_77e2daeb57dc4c1d9e46b4e05c32c6f2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:52.134065 4626 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet e4e5cb0d645846159e09ab71a897f6df
I20260927 23:28:52.137346 5095 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e5cb0d645846159e09ab71a897f6df with delete_type TABLET_DATA_DELETED (replaced by tablet e7d138c6dbe5403d9fc7277bd9bb4fce) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:52.144927 5221 tablet_bootstrap.cc:492] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:52.154004 4626 catalog_manager.cc:6565] ReplaceTablet: tablet e4e5cb0d645846159e09ab71a897f6df deleted and replaced by tablet e7d138c6dbe5403d9fc7277bd9bb4fce
I20260927 23:28:52.155473 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e5cb0d645846159e09ab71a897f6df with delete_type TABLET_DATA_DELETED (replaced by tablet e7d138c6dbe5403d9fc7277bd9bb4fce) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:52.156471 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e5cb0d645846159e09ab71a897f6df with delete_type TABLET_DATA_DELETED (replaced by tablet e7d138c6dbe5403d9fc7277bd9bb4fce) from {username='slave'} at 127.0.0.1:53202
W20260927 23:28:52.177361 4617 catalog_manager.cc:4808] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet e4e5cb0d645846159e09ab71a897f6df because the tablet was not found. No further retry: Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:28:52.187031 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet e4e5cb0d645846159e09ab71a897f6df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e4e5cb0d645846159e09ab71a897f6df already in progress: creating tablet
W20260927 23:28:52.214114 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet e4e5cb0d645846159e09ab71a897f6df because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e4e5cb0d645846159e09ab71a897f6df already in progress: creating tablet
I20260927 23:28:52.218092 4625 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_14122b0b934a4107ab5120525f201e5c"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:52.222888 4627 catalog_manager.cc:2148] created HMS catalog entry for table [id=0aa4192ae5f3479d9df14729b8604f92]
I20260927 23:28:52.253780 5221 tablet_bootstrap.cc:654] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:52.221575 5219 raft_consensus.cc:348] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:52.255301 5219 raft_consensus.cc:374] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:52.255671 5219 raft_consensus.cc:729] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:52.258936 4813 tablet_service.cc:1467] Processing CreateTablet for tablet e7d138c6dbe5403d9fc7277bd9bb4fce (DEFAULT_TABLE table=default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.259479 5221 log.cc:824] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d: Log is configured to *not* fsync() on all Append() calls
I20260927 23:28:52.264014 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d138c6dbe5403d9fc7277bd9bb4fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.265478 5219 consensus_queue.cc:260] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:52.275455 5098 tablet_service.cc:1467] Processing CreateTablet for tablet e7d138c6dbe5403d9fc7277bd9bb4fce (DEFAULT_TABLE table=default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.276875 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d138c6dbe5403d9fc7277bd9bb4fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.278651 4953 tablet_service.cc:1467] Processing CreateTablet for tablet e7d138c6dbe5403d9fc7277bd9bb4fce (DEFAULT_TABLE table=default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.280165 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7d138c6dbe5403d9fc7277bd9bb4fce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.282492 4627 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35136 (request call id 4) took 1055 ms. Trace:
I20260927 23:28:52.282944 4627 rpcz_store.cc:276] 0927 23:28:51.227341 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:51.227742 (+ 401us) service_pool.cc:227] Handling call
0927 23:28:51.866161 (+638419us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:51.868160 (+ 1999us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:28:52.222690 (+354530us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:28:52.281423 (+ 58733us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:28:52.281875 (+ 452us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:28:52.282357 (+ 482us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:28:52.225354 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:28:52.225843 (+ 489us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:28:52.225873 (+ 30us) write_op.cc:435] Acquired schema lock
0927 23:28:52.225889 (+ 16us) tablet.cc:660] Decoding operations
0927 23:28:52.226283 (+ 394us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:28:52.226328 (+ 45us) write_op.cc:641] Partition lock acquired for write op
0927 23:28:52.226360 (+ 32us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:28:52.226647 (+ 287us) tablet.cc:699] Row locks acquired
0927 23:28:52.226667 (+ 20us) write_op.cc:260] PREPARE: finished
0927 23:28:52.226822 (+ 155us) write_op.cc:270] Start()
0927 23:28:52.226927 (+ 105us) write_op.cc:276] Timestamp: P: 1790551732226761 usec, L: 0
0927 23:28:52.226947 (+ 20us) op_driver.cc:331] REPLICATION: starting
0927 23:28:52.227617 (+ 670us) log.cc:842] Serialized 432 byte log entry
0927 23:28:52.241872 (+ 14255us) op_driver.cc:444] REPLICATION: finished
0927 23:28:52.243862 (+ 1990us) write_op.cc:301] APPLY: starting
0927 23:28:52.243925 (+ 63us) tablet.cc:1365] starting BulkCheckPresence
0927 23:28:52.244047 (+ 122us) tablet.cc:1368] finished BulkCheckPresence
0927 23:28:52.244141 (+ 94us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:28:52.244616 (+ 475us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:28:52.244767 (+ 151us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:28:52.280542 (+ 35775us) write_op.cc:312] APPLY: finished
0927 23:28:52.280817 (+ 275us) log.cc:842] Serialized 28 byte log entry
0927 23:28:52.281016 (+ 199us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:52.281091 (+ 75us) write_op.cc:454] Released schema lock
0927 23:28:52.281132 (+ 41us) write_op.cc:341] FINISH: Updating metrics
0927 23:28:52.281359 (+ 227us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:28:52.281385 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":120128,"Hive Metastore.run_cpu_time_us":1025,"Hive Metastore.run_wall_time_us":233537,"child_traces":[["op",{"apply.queue_time_us":1834,"apply.run_cpu_time_us":2034,"apply.run_wall_time_us":37717,"num_ops":2,"prepare.queue_time_us":141,"prepare.run_cpu_time_us":2665,"prepare.run_wall_time_us":2664,"raft.queue_time_us":4180,"raft.run_cpu_time_us":932,"raft.run_wall_time_us":935,"replication_time_us":14842,"thread_start_us":11070,"threads_started":1}]]}
I20260927 23:28:52.291342 4626 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "default.Table_86bcd06c32004328b4e6830ef49b41c5" } modify_external_catalogs: true
I20260927 23:28:52.292341 4626 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "default.Table_86bcd06c32004328b4e6830ef49b41c5" } modify_external_catalogs: true
I20260927 23:28:52.296473 5219 ts_tablet_manager.cc:1428] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.200s user 0.037s sys 0.007s
I20260927 23:28:52.297564 5219 tablet_bootstrap.cc:492] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:52.311838 5219 tablet_bootstrap.cc:654] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:52.313088 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e5cb0d645846159e09ab71a897f6df with delete_type TABLET_DATA_DELETED (replaced by tablet e7d138c6dbe5403d9fc7277bd9bb4fce) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:52.314102 5224 tablet_replica.cc:330] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:52.315017 5224 raft_consensus.cc:2229] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:52.315697 5224 raft_consensus.cc:2258] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:52.318868 5224 ts_tablet_manager.cc:1905] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:52.339039 5219 tablet_bootstrap.cc:492] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:52.340124 5219 ts_tablet_manager.cc:1397] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.043s user 0.006s sys 0.005s
I20260927 23:28:52.343124 5219 raft_consensus.cc:348] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:52.343936 5219 raft_consensus.cc:374] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:52.352797 5224 ts_tablet_manager.cc:1918] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:52.353286 5224 log.cc:1196] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/e4e5cb0d645846159e09ab71a897f6df
I20260927 23:28:52.344235 5219 raft_consensus.cc:729] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:52.357151 5219 consensus_queue.cc:260] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:52.360812 5224 ts_tablet_manager.cc:1939] T e4e5cb0d645846159e09ab71a897f6df P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:52.362628 5219 ts_tablet_manager.cc:1428] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260927 23:28:52.366113 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet e4e5cb0d645846159e09ab71a897f6df (table default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:28:52.363590 5219 tablet_bootstrap.cc:492] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:52.377110 5096 tablet_service.cc:1467] Processing CreateTablet for tablet f171cc1f0094480a9cc6357eab93dff1 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.378376 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f171cc1f0094480a9cc6357eab93dff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.378472 4811 tablet_service.cc:1467] Processing CreateTablet for tablet f171cc1f0094480a9cc6357eab93dff1 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.379724 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f171cc1f0094480a9cc6357eab93dff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.381243 4954 tablet_service.cc:1467] Processing CreateTablet for tablet f171cc1f0094480a9cc6357eab93dff1 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.382439 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f171cc1f0094480a9cc6357eab93dff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.391199 5219 tablet_bootstrap.cc:654] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:52.392192 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 29eb6018ba1e424ba628221fe56a3cc1 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:52.419193 5217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:28:52.420185 5217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: This master's current role is: LEADER
I20260927 23:28:52.456873 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 29eb6018ba1e424ba628221fe56a3cc1 with delete_type TABLET_DATA_DELETED (replaced by tablet 9912e4dd70ba482398fbd831a8242f12) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:52.457773 5224 tablet_replica.cc:330] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:52.458437 5224 raft_consensus.cc:2229] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:52.458986 5224 raft_consensus.cc:2258] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:52.459049 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 29eb6018ba1e424ba628221fe56a3cc1
I20260927 23:28:52.461027 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 29eb6018ba1e424ba628221fe56a3cc1 with delete_type TABLET_DATA_DELETED (replaced by tablet 9912e4dd70ba482398fbd831a8242f12) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:52.461184 5224 ts_tablet_manager.cc:1905] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:52.466562 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 29eb6018ba1e424ba628221fe56a3cc1 with delete_type TABLET_DATA_DELETED (replaced by tablet 9912e4dd70ba482398fbd831a8242f12) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:52.468564 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 29eb6018ba1e424ba628221fe56a3cc1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 29eb6018ba1e424ba628221fe56a3cc1 already in progress: creating tablet
I20260927 23:28:52.469707 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet 29eb6018ba1e424ba628221fe56a3cc1 with delete_type TABLET_DATA_DELETED (replaced by tablet 9912e4dd70ba482398fbd831a8242f12) from {username='slave'} at 127.0.0.1:34620
W20260927 23:28:52.471572 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 29eb6018ba1e424ba628221fe56a3cc1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 29eb6018ba1e424ba628221fe56a3cc1 already in progress: creating tablet
I20260927 23:28:52.490115 5224 ts_tablet_manager.cc:1918] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:52.490587 5224 log.cc:1196] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/29eb6018ba1e424ba628221fe56a3cc1
I20260927 23:28:52.492585 4621 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_1c3db2bbcc0e488ea4b6637492ff4221"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:28:52.500815 5224 ts_tablet_manager.cc:1939] T 29eb6018ba1e424ba628221fe56a3cc1 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:52.502964 4954 tablet_service.cc:1467] Processing CreateTablet for tablet 9912e4dd70ba482398fbd831a8242f12 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.504500 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 29eb6018ba1e424ba628221fe56a3cc1 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:52.505019 4811 tablet_service.cc:1467] Processing CreateTablet for tablet 9912e4dd70ba482398fbd831a8242f12 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.505781 5098 tablet_service.cc:1467] Processing CreateTablet for tablet 9912e4dd70ba482398fbd831a8242f12 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.506160 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9912e4dd70ba482398fbd831a8242f12. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:52.506218 4614 catalog_manager.cc:4808] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet 29eb6018ba1e424ba628221fe56a3cc1 because the tablet was not found. No further retry: Not found: Tablet not found: 29eb6018ba1e424ba628221fe56a3cc1
I20260927 23:28:52.506918 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9912e4dd70ba482398fbd831a8242f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.504225 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9912e4dd70ba482398fbd831a8242f12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.513373 4622 catalog_manager.cc:6565] ReplaceTablet: tablet 29eb6018ba1e424ba628221fe56a3cc1 deleted and replaced by tablet 9912e4dd70ba482398fbd831a8242f12
I20260927 23:28:52.515558 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet baae829e8b9c4e7f8c2d3a1414666fcb from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:52.521205 5229 tablet_bootstrap.cc:492] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:52.543519 5221 tablet_bootstrap.cc:492] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:52.548337 5219 tablet_bootstrap.cc:492] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:52.548859 5219 ts_tablet_manager.cc:1397] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.185s user 0.028s sys 0.048s
I20260927 23:28:52.544188 5221 ts_tablet_manager.cc:1397] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.400s user 0.029s sys 0.019s
I20260927 23:28:52.551882 5219 raft_consensus.cc:348] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:52.552702 5219 raft_consensus.cc:374] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:52.552974 5219 raft_consensus.cc:729] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:52.553649 5219 consensus_queue.cc:260] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:52.562367 5219 ts_tablet_manager.cc:1428] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20260927 23:28:52.567348 5219 tablet_bootstrap.cc:492] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:52.574591 5219 tablet_bootstrap.cc:654] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:52.586917 5219 tablet_bootstrap.cc:492] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:52.587483 5219 ts_tablet_manager.cc:1397] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.009s
I20260927 23:28:52.590616 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:28:52.593251 5219 raft_consensus.cc:348] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:52.594013 5219 raft_consensus.cc:374] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:52.594202 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet baae829e8b9c4e7f8c2d3a1414666fcb with delete_type TABLET_DATA_DELETED (replaced by tablet 0c7f48abe7db4a5aa39892af9a3e3713) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:52.594305 5219 raft_consensus.cc:729] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:52.594996 5224 tablet_replica.cc:330] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:52.595711 5224 raft_consensus.cc:2229] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:52.596220 5224 raft_consensus.cc:2258] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:52.599884 5224 ts_tablet_manager.cc:1905] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:52.611732 5219 consensus_queue.cc:260] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:52.620986 5219 ts_tablet_manager.cc:1428] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.033s user 0.006s sys 0.000s
I20260927 23:28:52.629431 5219 tablet_bootstrap.cc:492] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:52.641083 5219 tablet_bootstrap.cc:654] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:52.648036 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet baae829e8b9c4e7f8c2d3a1414666fcb with delete_type TABLET_DATA_DELETED (replaced by tablet 0c7f48abe7db4a5aa39892af9a3e3713) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:52.649493 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet baae829e8b9c4e7f8c2d3a1414666fcb with delete_type TABLET_DATA_DELETED (replaced by tablet 0c7f48abe7db4a5aa39892af9a3e3713) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:52.650858 4627 catalog_manager.cc:6565] ReplaceTablet: tablet baae829e8b9c4e7f8c2d3a1414666fcb deleted and replaced by tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:28:52.651028 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet baae829e8b9c4e7f8c2d3a1414666fcb with delete_type TABLET_DATA_DELETED (replaced by tablet 0c7f48abe7db4a5aa39892af9a3e3713) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:52.653090 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet baae829e8b9c4e7f8c2d3a1414666fcb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet baae829e8b9c4e7f8c2d3a1414666fcb already in progress: creating tablet
W20260927 23:28:52.653829 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet baae829e8b9c4e7f8c2d3a1414666fcb because tablet deleting was already in progress. No further retry: Already present: State transition of tablet baae829e8b9c4e7f8c2d3a1414666fcb already in progress: creating tablet
I20260927 23:28:52.660621 5224 ts_tablet_manager.cc:1918] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:52.661072 5224 log.cc:1196] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:28:52.661752 5224 ts_tablet_manager.cc:1939] T baae829e8b9c4e7f8c2d3a1414666fcb P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:52.659850 5221 raft_consensus.cc:348] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:52.666337 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet baae829e8b9c4e7f8c2d3a1414666fcb (table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]) successfully deleted
I20260927 23:28:52.666626 5221 raft_consensus.cc:374] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:52.666550 4813 tablet_service.cc:1467] Processing CreateTablet for tablet 0c7f48abe7db4a5aa39892af9a3e3713 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.666936 5221 raft_consensus.cc:729] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:52.666824 5096 tablet_service.cc:1467] Processing CreateTablet for tablet 0c7f48abe7db4a5aa39892af9a3e3713 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:52.667157 4614 catalog_manager.cc:4808] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet baae829e8b9c4e7f8c2d3a1414666fcb because the tablet was not found. No further retry: Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:28:52.667788 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c7f48abe7db4a5aa39892af9a3e3713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.667992 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c7f48abe7db4a5aa39892af9a3e3713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.668037 5221 consensus_queue.cc:260] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:52.670933 4954 tablet_service.cc:1467] Processing CreateTablet for tablet 0c7f48abe7db4a5aa39892af9a3e3713 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.672158 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c7f48abe7db4a5aa39892af9a3e3713. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.695937 5229 tablet_bootstrap.cc:654] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:52.696451 5219 tablet_bootstrap.cc:492] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:52.696931 5219 ts_tablet_manager.cc:1397] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.068s user 0.010s sys 0.010s
I20260927 23:28:52.698978 5229 log.cc:824] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a: Log is configured to *not* fsync() on all Append() calls
I20260927 23:28:52.699690 5219 raft_consensus.cc:348] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:52.700455 5219 raft_consensus.cc:374] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:52.700726 5219 raft_consensus.cc:729] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:52.712759 5219 consensus_queue.cc:260] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:52.722491 5221 ts_tablet_manager.cc:1428] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.170s user 0.041s sys 0.010s
I20260927 23:28:52.723613 5221 tablet_bootstrap.cc:492] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:52.730942 4886 tablet.cc:2369] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:28:52.736600 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet f171cc1f0094480a9cc6357eab93dff1 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:52.740176 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet baae829e8b9c4e7f8c2d3a1414666fcb with delete_type TABLET_DATA_DELETED (replaced by tablet 0c7f48abe7db4a5aa39892af9a3e3713) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:52.741171 5223 tablet_replica.cc:330] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:28:52.742046 5223 raft_consensus.cc:2229] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:52.742683 5223 raft_consensus.cc:2258] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:52.745944 5223 ts_tablet_manager.cc:1905] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:52.749601 5221 tablet_bootstrap.cc:654] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:52.755899 5219 ts_tablet_manager.cc:1428] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.058s user 0.004s sys 0.003s
I20260927 23:28:52.760596 5219 tablet_bootstrap.cc:492] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:52.766664 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet f171cc1f0094480a9cc6357eab93dff1
I20260927 23:28:52.768873 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet f171cc1f0094480a9cc6357eab93dff1 with delete_type TABLET_DATA_DELETED (replaced by tablet e0f9998e6eb24f0a8fa217985ce3b07d) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:52.770278 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet f171cc1f0094480a9cc6357eab93dff1 with delete_type TABLET_DATA_DELETED (replaced by tablet e0f9998e6eb24f0a8fa217985ce3b07d) from {username='slave'} at 127.0.0.1:53202
W20260927 23:28:52.770675 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet f171cc1f0094480a9cc6357eab93dff1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f171cc1f0094480a9cc6357eab93dff1 already in progress: creating tablet
I20260927 23:28:52.771044 5224 tablet_replica.cc:330] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:52.771819 5224 raft_consensus.cc:2229] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:52.770542 4622 catalog_manager.cc:6565] ReplaceTablet: tablet f171cc1f0094480a9cc6357eab93dff1 deleted and replaced by tablet e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:28:52.772346 5224 raft_consensus.cc:2258] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:52.772408 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet f171cc1f0094480a9cc6357eab93dff1 with delete_type TABLET_DATA_DELETED (replaced by tablet e0f9998e6eb24f0a8fa217985ce3b07d) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:52.774762 5221 tablet_bootstrap.cc:492] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:52.775233 5221 ts_tablet_manager.cc:1397] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.052s user 0.010s sys 0.000s
I20260927 23:28:52.775302 5224 ts_tablet_manager.cc:1905] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:52.778095 5221 raft_consensus.cc:348] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:52.778829 5221 raft_consensus.cc:374] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:52.779129 5221 raft_consensus.cc:729] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:52.779855 5221 consensus_queue.cc:260] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:52.792233 5229 tablet_bootstrap.cc:492] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:52.792965 5229 ts_tablet_manager.cc:1397] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.272s user 0.015s sys 0.003s
I20260927 23:28:52.793206 5221 ts_tablet_manager.cc:1428] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.018s user 0.008s sys 0.000s
I20260927 23:28:52.794122 5221 tablet_bootstrap.cc:492] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:52.803120 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet e4e5cb0d645846159e09ab71a897f6df with delete_type TABLET_DATA_DELETED (replaced by tablet e7d138c6dbe5403d9fc7277bd9bb4fce) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:52.811232 5219 tablet_bootstrap.cc:654] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:52.825438 5221 tablet_bootstrap.cc:654] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:52.837854 4813 tablet_service.cc:1467] Processing CreateTablet for tablet e0f9998e6eb24f0a8fa217985ce3b07d (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.838347 4954 tablet_service.cc:1467] Processing CreateTablet for tablet e0f9998e6eb24f0a8fa217985ce3b07d (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.839254 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f9998e6eb24f0a8fa217985ce3b07d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.839557 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f9998e6eb24f0a8fa217985ce3b07d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.840955 5221 tablet_bootstrap.cc:492] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:52.841439 5221 ts_tablet_manager.cc:1397] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.047s user 0.013s sys 0.001s
I20260927 23:28:52.844507 5221 raft_consensus.cc:348] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:52.845330 5221 raft_consensus.cc:374] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:52.845628 5221 raft_consensus.cc:729] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:52.845891 5224 ts_tablet_manager.cc:1918] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:52.846311 5224 log.cc:1196] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/f171cc1f0094480a9cc6357eab93dff1
I20260927 23:28:52.846962 5224 ts_tablet_manager.cc:1939] T f171cc1f0094480a9cc6357eab93dff1 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:52.848888 5223 ts_tablet_manager.cc:1918] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:52.849530 5223 log.cc:1196] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:28:52.850566 5223 ts_tablet_manager.cc:1939] T baae829e8b9c4e7f8c2d3a1414666fcb P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:28:52.851477 5221 consensus_queue.cc:260] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:52.853693 5221 ts_tablet_manager.cc:1428] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260927 23:28:52.855856 5221 tablet_bootstrap.cc:492] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:52.860642 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 29eb6018ba1e424ba628221fe56a3cc1 with delete_type TABLET_DATA_DELETED (replaced by tablet 9912e4dd70ba482398fbd831a8242f12) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:52.866684 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 9912e4dd70ba482398fbd831a8242f12 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:52.869602 5219 tablet_bootstrap.cc:492] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:52.869809 5096 tablet_service.cc:1467] Processing CreateTablet for tablet e0f9998e6eb24f0a8fa217985ce3b07d (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:52.870245 5026 tablet.cc:2369] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:28:52.870275 5219 ts_tablet_manager.cc:1397] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.110s user 0.006s sys 0.007s
I20260927 23:28:52.873893 5221 tablet_bootstrap.cc:654] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:52.875891 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e0f9998e6eb24f0a8fa217985ce3b07d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.873111 5219 raft_consensus.cc:348] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:52.878767 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet f171cc1f0094480a9cc6357eab93dff1 (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
I20260927 23:28:52.878971 5219 raft_consensus.cc:374] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:52.880625 5219 raft_consensus.cc:729] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
W20260927 23:28:52.881481 5209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260927 23:28:52.878597 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet baae829e8b9c4e7f8c2d3a1414666fcb (table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]) successfully deleted
W20260927 23:28:52.883087 5209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
I20260927 23:28:52.883975 5223 tablet_replica.cc:330] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:28:52.882696 5219 consensus_queue.cc:260] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:52.883592 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet f171cc1f0094480a9cc6357eab93dff1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet f171cc1f0094480a9cc6357eab93dff1 already in progress: creating tablet
W20260927 23:28:52.885174 5209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260927 23:28:52.885175 5223 raft_consensus.cc:2229] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:52.885667 5223 raft_consensus.cc:2258] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:52.887980 5223 ts_tablet_manager.cc:1905] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:52.913331 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 9912e4dd70ba482398fbd831a8242f12
I20260927 23:28:52.915987 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet 9912e4dd70ba482398fbd831a8242f12 with delete_type TABLET_DATA_DELETED (replaced by tablet 14b3852ae71c48fc830ec6939a53c18a) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:52.917570 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 9912e4dd70ba482398fbd831a8242f12 with delete_type TABLET_DATA_DELETED (replaced by tablet 14b3852ae71c48fc830ec6939a53c18a) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:52.919085 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 9912e4dd70ba482398fbd831a8242f12 with delete_type TABLET_DATA_DELETED (replaced by tablet 14b3852ae71c48fc830ec6939a53c18a) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:52.920408 5224 tablet_replica.cc:330] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:52.921123 5224 raft_consensus.cc:2229] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:52.921308 4622 catalog_manager.cc:6565] ReplaceTablet: tablet 9912e4dd70ba482398fbd831a8242f12 deleted and replaced by tablet 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:28:52.921633 5224 raft_consensus.cc:2258] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:52.921695 5221 tablet_bootstrap.cc:492] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:52.922376 5221 ts_tablet_manager.cc:1397] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.067s user 0.013s sys 0.012s
I20260927 23:28:52.924041 5224 ts_tablet_manager.cc:1905] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:52.924440 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 9912e4dd70ba482398fbd831a8242f12 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9912e4dd70ba482398fbd831a8242f12 already in progress: creating tablet
I20260927 23:28:52.926420 5219 ts_tablet_manager.cc:1428] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.056s user 0.006s sys 0.000s
I20260927 23:28:52.932751 5223 ts_tablet_manager.cc:1918] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:52.934000 5221 raft_consensus.cc:348] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:52.944689 5221 raft_consensus.cc:374] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:52.944881 5219 tablet_bootstrap.cc:492] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:52.945152 5221 raft_consensus.cc:729] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:52.946054 5221 consensus_queue.cc:260] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:52.948372 5221 ts_tablet_manager.cc:1428] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.026s user 0.003s sys 0.004s
I20260927 23:28:52.949120 5223 log.cc:1196] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/e4e5cb0d645846159e09ab71a897f6df
I20260927 23:28:52.950877 5223 ts_tablet_manager.cc:1939] T e4e5cb0d645846159e09ab71a897f6df P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:28:52.953552 5223 tablet_replica.cc:330] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:28:52.954381 5223 raft_consensus.cc:2229] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:52.950109 5221 tablet_bootstrap.cc:492] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:52.960893 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet e4e5cb0d645846159e09ab71a897f6df (table default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:28:52.961711 5223 raft_consensus.cc:2258] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:52.964183 5223 ts_tablet_manager.cc:1905] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:52.964154 5229 raft_consensus.cc:348] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:52.966029 5096 tablet_service.cc:1467] Processing CreateTablet for tablet 14b3852ae71c48fc830ec6939a53c18a (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.967856 4954 tablet_service.cc:1467] Processing CreateTablet for tablet 14b3852ae71c48fc830ec6939a53c18a (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.969542 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14b3852ae71c48fc830ec6939a53c18a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.966902 5221 tablet_bootstrap.cc:654] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:52.968585 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet f171cc1f0094480a9cc6357eab93dff1 with delete_type TABLET_DATA_DELETED (replaced by tablet e0f9998e6eb24f0a8fa217985ce3b07d) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:52.980135 4813 tablet_service.cc:1467] Processing CreateTablet for tablet 14b3852ae71c48fc830ec6939a53c18a (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:52.981389 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14b3852ae71c48fc830ec6939a53c18a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:52.981952 5231 raft_consensus.cc:482] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:52.982514 5231 raft_consensus.cc:504] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:52.986737 5231 leader_election.cc:288] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:52.990669 5224 ts_tablet_manager.cc:1918] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:52.991065 5224 log.cc:1196] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/9912e4dd70ba482398fbd831a8242f12
I20260927 23:28:52.991765 5224 ts_tablet_manager.cc:1939] T 9912e4dd70ba482398fbd831a8242f12 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:52.992620 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 0c7f48abe7db4a5aa39892af9a3e3713 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:52.994601 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 9912e4dd70ba482398fbd831a8242f12 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:53.001683 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14b3852ae71c48fc830ec6939a53c18a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:53.006994 5229 raft_consensus.cc:374] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.007475 5229 raft_consensus.cc:729] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:53.011127 5219 tablet_bootstrap.cc:654] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.027753 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:28:53.029698 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c7f48abe7db4a5aa39892af9a3e3713 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a0c6ad197164aa98d73d357de26b488) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:53.035378 5209 flags.cc:425] Enabled experimental flag: --enable_metadata_cleanup_for_deleted_tables_and_tablets=true
W20260927 23:28:53.035765 5209 flags.cc:425] Enabled experimental flag: --metadata_for_deleted_table_and_tablet_reserved_secs=0
W20260927 23:28:53.036152 5209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 23:28:53.036593 5209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 23:28:53.036952 5209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 23:28:53.037295 5209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 23:28:53.041162 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 0c7f48abe7db4a5aa39892af9a3e3713 deleted and replaced by tablet 3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:53.042006 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c7f48abe7db4a5aa39892af9a3e3713 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a0c6ad197164aa98d73d357de26b488) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:53.018357 5229 consensus_queue.cc:260] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:53.043848 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet 0c7f48abe7db4a5aa39892af9a3e3713 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c7f48abe7db4a5aa39892af9a3e3713 already in progress: creating tablet
W20260927 23:28:53.032038 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 0c7f48abe7db4a5aa39892af9a3e3713 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c7f48abe7db4a5aa39892af9a3e3713 already in progress: creating tablet
I20260927 23:28:53.060905 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c7f48abe7db4a5aa39892af9a3e3713 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a0c6ad197164aa98d73d357de26b488) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:53.070174 5229 ts_tablet_manager.cc:1428] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.277s user 0.033s sys 0.007s
I20260927 23:28:53.073670 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 29eb6018ba1e424ba628221fe56a3cc1 with delete_type TABLET_DATA_DELETED (replaced by tablet 9912e4dd70ba482398fbd831a8242f12) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:53.074636 5222 tablet_replica.cc:330] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a: stopping tablet replica
I20260927 23:28:53.075464 5222 raft_consensus.cc:2229] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:53.076061 5222 raft_consensus.cc:2258] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:53.076056 5229 tablet_bootstrap.cc:492] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:53.080328 5221 tablet_bootstrap.cc:492] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:53.080840 5221 ts_tablet_manager.cc:1397] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.131s user 0.016s sys 0.014s
I20260927 23:28:53.085606 5222 ts_tablet_manager.cc:1905] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:53.099324 5223 ts_tablet_manager.cc:1918] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:53.099865 5223 log.cc:1196] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/29eb6018ba1e424ba628221fe56a3cc1
I20260927 23:28:53.083545 5221 raft_consensus.cc:348] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:53.102516 5221 raft_consensus.cc:374] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.102829 5221 raft_consensus.cc:729] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:53.105091 5229 tablet_bootstrap.cc:654] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.106638 5221 consensus_queue.cc:260] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:53.110323 5219 tablet_bootstrap.cc:492] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:53.110831 5219 ts_tablet_manager.cc:1397] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.166s user 0.009s sys 0.002s
I20260927 23:28:53.111155 5223 ts_tablet_manager.cc:1939] T 29eb6018ba1e424ba628221fe56a3cc1 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:28:53.113456 5219 raft_consensus.cc:348] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:53.114144 5219 raft_consensus.cc:374] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.114389 5219 raft_consensus.cc:729] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:53.116645 5219 consensus_queue.cc:260] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:53.122630 5209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.57.20:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_preallocate_segments=false
--leader_failure_max_missed_heartbeat_periods=1
--fs_data_dirs=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/wal
--hive_metastore_uris=thrift://127.0.0.1:36031
--catalog_manager_bg_task_wait_ms=10
--enable_metadata_cleanup_for_deleted_tables_and_tablets=true
--metadata_for_deleted_table_and_tablet_reserved_secs=0
--hive_metastore_notification_log_poll_period_seconds=1
--ipki_ca_key_size=768
--master_addresses=127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.57.62:35155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.57.62
--webserver_port=34737
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 27 Sep 2026 22:42:10 UTC on ccycloud.quasar-yawxpd.root.comops.site
build id 4676391
TSAN enabled
I20260927 23:28:53.125149 5209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 23:28:53.128101 5209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 23:28:53.133211 5219 ts_tablet_manager.cc:1428] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.022s user 0.007s sys 0.001s
I20260927 23:28:53.134334 5219 tablet_bootstrap.cc:492] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:53.143206 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c7f48abe7db4a5aa39892af9a3e3713 with delete_type TABLET_DATA_DELETED (replaced by tablet 3a0c6ad197164aa98d73d357de26b488) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:53.144122 5224 tablet_replica.cc:330] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:53.144928 5224 raft_consensus.cc:2229] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:53.145426 5224 raft_consensus.cc:2258] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:53.147818 5224 ts_tablet_manager.cc:1905] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:53.158362 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet e7d138c6dbe5403d9fc7277bd9bb4fce from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:53.168521 5222 ts_tablet_manager.cc:1918] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260927 23:28:53.170825 5258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 23:28:53.171092 5222 log.cc:1196] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/29eb6018ba1e424ba628221fe56a3cc1
I20260927 23:28:53.177964 5222 ts_tablet_manager.cc:1939] T 29eb6018ba1e424ba628221fe56a3cc1 P f5ec98b2972d44838fa273eb7a10d01a: Deleting consensus metadata
I20260927 23:28:53.179245 5219 tablet_bootstrap.cc:654] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.180166 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7d138c6dbe5403d9fc7277bd9bb4fce" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:53.192035 4954 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0c6ad197164aa98d73d357de26b488 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:28:53.182544 4769 leader_election.cc:341] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:53.191053 4811 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0c6ad197164aa98d73d357de26b488 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:53.193387 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a0c6ad197164aa98d73d357de26b488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:53.196877 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a0c6ad197164aa98d73d357de26b488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:53.203773 5240 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:53.204991 4617 catalog_manager.cc:4841] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): tablet 29eb6018ba1e424ba628221fe56a3cc1 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:53.204528 5240 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:53.205112 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7d138c6dbe5403d9fc7277bd9bb4fce" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:53.208186 4769 leader_election.cc:341] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:53.208712 4769 leader_election.cc:302] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:53.209717 5231 raft_consensus.cc:2726] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:53.213069 5240 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
I20260927 23:28:53.222615 5224 ts_tablet_manager.cc:1918] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:53.229786 5224 log.cc:1196] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:28:53.230949 5224 ts_tablet_manager.cc:1939] T 0c7f48abe7db4a5aa39892af9a3e3713 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:53.234965 5219 tablet_bootstrap.cc:492] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:53.235551 5219 ts_tablet_manager.cc:1397] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.101s user 0.008s sys 0.005s
I20260927 23:28:53.236431 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet e7d138c6dbe5403d9fc7277bd9bb4fce
I20260927 23:28:53.238404 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet e7d138c6dbe5403d9fc7277bd9bb4fce with delete_type TABLET_DATA_DELETED (replaced by tablet 0a3148da66024c4684d5eaf3d60c8c58) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:53.240095 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet e7d138c6dbe5403d9fc7277bd9bb4fce with delete_type TABLET_DATA_DELETED (replaced by tablet 0a3148da66024c4684d5eaf3d60c8c58) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:53.240878 4622 catalog_manager.cc:6565] ReplaceTablet: tablet e7d138c6dbe5403d9fc7277bd9bb4fce deleted and replaced by tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:28:53.242102 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet e7d138c6dbe5403d9fc7277bd9bb4fce because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7d138c6dbe5403d9fc7277bd9bb4fce already in progress: creating tablet
I20260927 23:28:53.240092 5219 raft_consensus.cc:348] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:53.244957 5219 raft_consensus.cc:374] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.245298 5219 raft_consensus.cc:729] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:53.246034 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet e7d138c6dbe5403d9fc7277bd9bb4fce with delete_type TABLET_DATA_DELETED (replaced by tablet 0a3148da66024c4684d5eaf3d60c8c58) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:53.249995 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 0c7f48abe7db4a5aa39892af9a3e3713 (table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]) successfully deleted
I20260927 23:28:53.248910 5219 consensus_queue.cc:260] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:53.250854 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet e7d138c6dbe5403d9fc7277bd9bb4fce from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:53.251325 5224 tablet_replica.cc:330] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:53.252068 5224 raft_consensus.cc:2229] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:53.257682 5219 ts_tablet_manager.cc:1428] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260927 23:28:53.260648 5224 raft_consensus.cc:2258] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:53.267716 5219 tablet_bootstrap.cc:492] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
W20260927 23:28:53.270102 4907 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:28:53.276872 5224 ts_tablet_manager.cc:1905] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:53.291308 5240 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:53.291851 5240 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:53.295892 5240 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:28:53.297762 4907 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:28:53.301677 5219 tablet_bootstrap.cc:654] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.310842 5096 tablet_service.cc:1467] Processing CreateTablet for tablet 3a0c6ad197164aa98d73d357de26b488 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:53.312223 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a0c6ad197164aa98d73d357de26b488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:53.319880 5221 ts_tablet_manager.cc:1428] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.239s user 0.006s sys 0.002s
I20260927 23:28:53.330360 5219 tablet_bootstrap.cc:492] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:53.330258 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet 29eb6018ba1e424ba628221fe56a3cc1 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:53.330857 5219 ts_tablet_manager.cc:1397] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.063s user 0.011s sys 0.000s
I20260927 23:28:53.331516 5223 tablet_replica.cc:330] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:28:53.332455 5223 raft_consensus.cc:2229] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:53.334964 5219 raft_consensus.cc:348] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:53.335842 5219 raft_consensus.cc:374] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.341862 5221 tablet_bootstrap.cc:492] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:53.336109 5219 raft_consensus.cc:729] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:53.345077 5219 consensus_queue.cc:260] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:53.351210 5219 ts_tablet_manager.cc:1428] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.020s user 0.008s sys 0.000s
I20260927 23:28:53.359228 5224 ts_tablet_manager.cc:1918] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:53.333055 5223 raft_consensus.cc:2258] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:53.378916 5229 tablet_bootstrap.cc:492] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:53.379477 5229 ts_tablet_manager.cc:1397] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.304s user 0.008s sys 0.004s
I20260927 23:28:53.382565 5229 raft_consensus.cc:348] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:53.383533 5229 raft_consensus.cc:374] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.362622 5224 log.cc:1196] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/e7d138c6dbe5403d9fc7277bd9bb4fce
I20260927 23:28:53.392833 5224 ts_tablet_manager.cc:1939] T e7d138c6dbe5403d9fc7277bd9bb4fce P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:53.396061 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet e7d138c6dbe5403d9fc7277bd9bb4fce (table default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
W20260927 23:28:53.401036 4685 hms_client.cc:336] Time spent get HMS table: real 1.147s user 0.000s sys 0.001s
I20260927 23:28:53.401053 5223 ts_tablet_manager.cc:1905] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:53.404682 5231 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:53.405272 5231 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:53.407534 5231 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:53.409670 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f9998e6eb24f0a8fa217985ce3b07d" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:28:53.412462 4769 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:53.390686 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f171cc1f0094480a9cc6357eab93dff1" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:53.391636 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9912e4dd70ba482398fbd831a8242f12" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:53.411202 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e0f9998e6eb24f0a8fa217985ce3b07d" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:53.419468 4910 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:53.419957 4910 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:53.420589 5229 raft_consensus.cc:729] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
W20260927 23:28:53.420795 4769 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:53.421218 4769 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
W20260927 23:28:53.421824 4910 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:53.421602 5229 consensus_queue.cc:260] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:53.422260 5231 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:53.422209 4910 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
W20260927 23:28:53.422721 4910 raft_consensus.cc:2670] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260927 23:28:53.423435 5240 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:53.449689 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 14b3852ae71c48fc830ec6939a53c18a from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:53.452174 5219 tablet_bootstrap.cc:492] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:53.458359 5219 tablet_bootstrap.cc:654] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.459545 5221 tablet_bootstrap.cc:654] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.475238 5229 ts_tablet_manager.cc:1428] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.095s user 0.006s sys 0.001s
I20260927 23:28:53.475944 5223 ts_tablet_manager.cc:1918] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:53.480840 5223 log.cc:1196] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/9912e4dd70ba482398fbd831a8242f12
I20260927 23:28:53.481719 5223 ts_tablet_manager.cc:1939] T 9912e4dd70ba482398fbd831a8242f12 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:28:53.484609 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet 9912e4dd70ba482398fbd831a8242f12 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:53.492041 5223 tablet_replica.cc:330] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:28:53.492838 5223 raft_consensus.cc:2229] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:53.493393 5223 raft_consensus.cc:2258] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:53.502663 5223 ts_tablet_manager.cc:1905] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:53.503153 5219 tablet_bootstrap.cc:492] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:53.504989 5219 ts_tablet_manager.cc:1397] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.053s user 0.013s sys 0.003s
I20260927 23:28:53.506206 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:28:53.507380 5219 raft_consensus.cc:348] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:53.508062 5219 raft_consensus.cc:374] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.508327 5219 raft_consensus.cc:729] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:53.508924 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 14b3852ae71c48fc830ec6939a53c18a deleted and replaced by tablet 43f8e7644db24be5a758b437a5f471c0
I20260927 23:28:53.508930 5219 consensus_queue.cc:260] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:53.509519 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet 14b3852ae71c48fc830ec6939a53c18a with delete_type TABLET_DATA_DELETED (replaced by tablet 43f8e7644db24be5a758b437a5f471c0) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:53.512003 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 14b3852ae71c48fc830ec6939a53c18a with delete_type TABLET_DATA_DELETED (replaced by tablet 43f8e7644db24be5a758b437a5f471c0) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:53.514138 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 14b3852ae71c48fc830ec6939a53c18a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14b3852ae71c48fc830ec6939a53c18a already in progress: creating tablet
I20260927 23:28:53.516709 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 14b3852ae71c48fc830ec6939a53c18a with delete_type TABLET_DATA_DELETED (replaced by tablet 43f8e7644db24be5a758b437a5f471c0) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:53.517474 5224 tablet_replica.cc:330] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:53.519446 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 3a0c6ad197164aa98d73d357de26b488 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:53.521416 4811 tablet_service.cc:1467] Processing CreateTablet for tablet 0a3148da66024c4684d5eaf3d60c8c58 (DEFAULT_TABLE table=default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:53.522146 5224 raft_consensus.cc:2229] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:53.522702 5224 raft_consensus.cc:2258] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:53.523010 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a3148da66024c4684d5eaf3d60c8c58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:53.524921 5098 tablet_service.cc:1467] Processing CreateTablet for tablet 0a3148da66024c4684d5eaf3d60c8c58 (DEFAULT_TABLE table=default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:53.525175 5224 ts_tablet_manager.cc:1905] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:53.526108 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a3148da66024c4684d5eaf3d60c8c58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:53.532636 5219 ts_tablet_manager.cc:1428] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.027s user 0.006s sys 0.001s
I20260927 23:28:53.534401 5229 tablet_bootstrap.cc:492] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
W20260927 23:28:53.553776 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:28:53.555531 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:53.557581 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0c6ad197164aa98d73d357de26b488 with delete_type TABLET_DATA_DELETED (replaced by tablet e7a73bf1fc35439ea3e39ffd843ebfb1) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:53.559351 4617 catalog_manager.cc:4808] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 3a0c6ad197164aa98d73d357de26b488 because the tablet was not found. No further retry: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:53.561714 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0c6ad197164aa98d73d357de26b488 with delete_type TABLET_DATA_DELETED (replaced by tablet e7a73bf1fc35439ea3e39ffd843ebfb1) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:53.561919 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a0c6ad197164aa98d73d357de26b488 with delete_type TABLET_DATA_DELETED (replaced by tablet e7a73bf1fc35439ea3e39ffd843ebfb1) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:53.562738 4622 catalog_manager.cc:6565] ReplaceTablet: tablet 3a0c6ad197164aa98d73d357de26b488 deleted and replaced by tablet e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:28:53.566176 5221 tablet_bootstrap.cc:492] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:53.566717 5221 ts_tablet_manager.cc:1397] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.225s user 0.007s sys 0.003s
W20260927 23:28:53.583117 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:28:53.596553 5166 tablet.cc:2369] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 23:28:53.600524 4954 tablet_service.cc:1467] Processing CreateTablet for tablet 0a3148da66024c4684d5eaf3d60c8c58 (DEFAULT_TABLE table=default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:53.601862 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a3148da66024c4684d5eaf3d60c8c58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:53.603901 5229 tablet_bootstrap.cc:654] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.605556 5221 raft_consensus.cc:348] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:53.607177 5221 raft_consensus.cc:374] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.607491 5221 raft_consensus.cc:729] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:53.608170 5221 consensus_queue.cc:260] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:53.626982 5223 ts_tablet_manager.cc:1918] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:53.643915 5224 ts_tablet_manager.cc:1918] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:53.648622 5224 log.cc:1196] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/14b3852ae71c48fc830ec6939a53c18a
I20260927 23:28:53.649166 5223 log.cc:1196] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/f171cc1f0094480a9cc6357eab93dff1
I20260927 23:28:53.649394 5224 ts_tablet_manager.cc:1939] T 14b3852ae71c48fc830ec6939a53c18a P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:53.652139 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 14b3852ae71c48fc830ec6939a53c18a (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:53.656314 5223 ts_tablet_manager.cc:1939] T f171cc1f0094480a9cc6357eab93dff1 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:28:53.659340 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet f171cc1f0094480a9cc6357eab93dff1 (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
I20260927 23:28:53.634470 5221 ts_tablet_manager.cc:1428] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.067s user 0.008s sys 0.000s
I20260927 23:28:53.662319 5224 tablet_replica.cc:330] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:53.665536 5224 raft_consensus.cc:2229] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:53.671245 5221 tablet_bootstrap.cc:492] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:53.672096 5224 raft_consensus.cc:2258] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:53.676280 5224 ts_tablet_manager.cc:1905] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:53.697558 5223 tablet_replica.cc:330] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
W20260927 23:28:53.697865 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 0c7f48abe7db4a5aa39892af9a3e3713 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c7f48abe7db4a5aa39892af9a3e3713 already in progress: creating tablet
I20260927 23:28:53.698328 5223 raft_consensus.cc:2229] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:53.698810 5223 raft_consensus.cc:2258] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:53.699010 5252 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:53.699623 5252 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:53.702638 5221 tablet_bootstrap.cc:654] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.707463 5252 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:53.707340 4813 tablet_service.cc:1467] Processing CreateTablet for tablet e7a73bf1fc35439ea3e39ffd843ebfb1 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:53.708613 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7a73bf1fc35439ea3e39ffd843ebfb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:53.716104 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 3a0c6ad197164aa98d73d357de26b488 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:53.719295 5223 ts_tablet_manager.cc:1905] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:53.723326 5229 tablet_bootstrap.cc:492] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:53.723868 5229 ts_tablet_manager.cc:1397] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.190s user 0.012s sys 0.000s
I20260927 23:28:53.709870 4954 tablet_service.cc:1467] Processing CreateTablet for tablet e7a73bf1fc35439ea3e39ffd843ebfb1 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:53.725754 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7a73bf1fc35439ea3e39ffd843ebfb1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:53.734944 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:28:53.739429 5219 tablet_bootstrap.cc:492] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:53.743080 4812 tablet_service.cc:1467] Processing CreateTablet for tablet 43f8e7644db24be5a758b437a5f471c0 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:53.744416 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43f8e7644db24be5a758b437a5f471c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:53.745644 5219 tablet_bootstrap.cc:654] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.753294 5096 tablet_service.cc:1467] Processing CreateTablet for tablet e7a73bf1fc35439ea3e39ffd843ebfb1 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:53.754577 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7a73bf1fc35439ea3e39ffd843ebfb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:53.758329 5229 raft_consensus.cc:348] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:53.759114 5229 raft_consensus.cc:374] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.759383 5229 raft_consensus.cc:729] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:53.768997 5219 tablet_bootstrap.cc:492] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:53.771402 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 0a3148da66024c4684d5eaf3d60c8c58 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:53.772166 5219 ts_tablet_manager.cc:1397] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.033s user 0.011s sys 0.002s
I20260927 23:28:53.776376 4953 tablet_service.cc:1467] Processing CreateTablet for tablet 43f8e7644db24be5a758b437a5f471c0 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:53.777624 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43f8e7644db24be5a758b437a5f471c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:53.779544 5219 raft_consensus.cc:348] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:53.785032 5219 raft_consensus.cc:374] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.790778 5219 raft_consensus.cc:729] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:53.792743 5219 consensus_queue.cc:260] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:53.802347 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 0a3148da66024c4684d5eaf3d60c8c58
I20260927 23:28:53.802733 5221 tablet_bootstrap.cc:492] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:53.803608 5221 ts_tablet_manager.cc:1397] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.133s user 0.007s sys 0.008s
I20260927 23:28:53.806348 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 0a3148da66024c4684d5eaf3d60c8c58 deleted and replaced by tablet ac88a4e4bee64586bef7b6b409a63f4c
I20260927 23:28:53.806882 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a3148da66024c4684d5eaf3d60c8c58 with delete_type TABLET_DATA_DELETED (replaced by tablet ac88a4e4bee64586bef7b6b409a63f4c) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:53.808158 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a3148da66024c4684d5eaf3d60c8c58 with delete_type TABLET_DATA_DELETED (replaced by tablet ac88a4e4bee64586bef7b6b409a63f4c) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:53.808156 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a3148da66024c4684d5eaf3d60c8c58 with delete_type TABLET_DATA_DELETED (replaced by tablet ac88a4e4bee64586bef7b6b409a63f4c) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:53.809751 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 0a3148da66024c4684d5eaf3d60c8c58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a3148da66024c4684d5eaf3d60c8c58 already in progress: creating tablet
I20260927 23:28:53.811492 5219 ts_tablet_manager.cc:1428] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.038s user 0.005s sys 0.001s
I20260927 23:28:53.818192 5098 tablet_service.cc:1467] Processing CreateTablet for tablet 43f8e7644db24be5a758b437a5f471c0 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:53.819427 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43f8e7644db24be5a758b437a5f471c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:53.825479 5219 tablet_bootstrap.cc:492] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:53.830040 5229 consensus_queue.cc:260] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:53.817818 5224 ts_tablet_manager.cc:1918] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:53.831254 5224 log.cc:1196] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:53.832283 5224 ts_tablet_manager.cc:1939] T 3a0c6ad197164aa98d73d357de26b488 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:53.833245 5221 raft_consensus.cc:348] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:53.834051 5221 raft_consensus.cc:374] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.834308 5221 raft_consensus.cc:729] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:53.835048 5221 consensus_queue.cc:260] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:53.837997 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 3a0c6ad197164aa98d73d357de26b488 (table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]) successfully deleted
I20260927 23:28:53.845685 5224 tablet_replica.cc:330] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:53.849545 5224 raft_consensus.cc:2229] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:53.851014 5224 raft_consensus.cc:2258] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:53.851440 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:28:53.851924 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:28:53.852401 5252 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:53.852843 5252 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:53.853843 5219 tablet_bootstrap.cc:654] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.857345 5224 ts_tablet_manager.cc:1905] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:53.862131 5252 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:53.862686 5252 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:53.862870 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:28:53.873238 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:28:53.878006 5219 tablet_bootstrap.cc:492] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:53.880137 5219 ts_tablet_manager.cc:1397] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.055s user 0.007s sys 0.004s
W20260927 23:28:53.885512 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:28:53.885982 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
W20260927 23:28:53.886281 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet e4e5cb0d645846159e09ab71a897f6df
I20260927 23:28:53.888924 5277 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:53.891129 5221 ts_tablet_manager.cc:1428] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.087s user 0.000s sys 0.007s
I20260927 23:28:53.883380 5219 raft_consensus.cc:348] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:53.893579 5219 raft_consensus.cc:374] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.894109 5229 ts_tablet_manager.cc:1428] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.170s user 0.009s sys 0.000s
I20260927 23:28:53.894526 5219 raft_consensus.cc:729] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
W20260927 23:28:53.899811 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:28:53.901149 5219 consensus_queue.cc:260] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:53.907047 5229 tablet_bootstrap.cc:492] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
W20260927 23:28:53.907707 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet e4e5cb0d645846159e09ab71a897f6df
I20260927 23:28:53.916100 5219 ts_tablet_manager.cc:1428] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.035s user 0.005s sys 0.000s
I20260927 23:28:53.916873 5229 tablet_bootstrap.cc:654] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.919319 5221 tablet_bootstrap.cc:492] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:53.934686 5221 tablet_bootstrap.cc:654] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.935389 5219 tablet_bootstrap.cc:492] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:53.937290 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 43f8e7644db24be5a758b437a5f471c0 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:53.945820 5219 tablet_bootstrap.cc:654] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.962669 5221 tablet_bootstrap.cc:492] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:53.963020 4630 catalog_manager.cc:3411] renamed table default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d] in HMS: new name default.table_e141ccac1ae746388643f3ee5f04cdc1
I20260927 23:28:53.963328 5221 ts_tablet_manager.cc:1397] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.044s user 0.010s sys 0.001s
I20260927 23:28:53.952642 5224 ts_tablet_manager.cc:1918] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:53.965004 5224 log.cc:1196] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/0a3148da66024c4684d5eaf3d60c8c58
I20260927 23:28:53.965982 5224 ts_tablet_manager.cc:1939] T 0a3148da66024c4684d5eaf3d60c8c58 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:53.965925 5221 raft_consensus.cc:348] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:53.966614 5221 raft_consensus.cc:374] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.966853 5221 raft_consensus.cc:729] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:53.967540 5221 consensus_queue.cc:260] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:53.976020 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 0a3148da66024c4684d5eaf3d60c8c58 (table default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:28:53.978832 5219 tablet_bootstrap.cc:492] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:53.979992 5221 ts_tablet_manager.cc:1428] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.016s user 0.005s sys 0.000s
I20260927 23:28:53.980657 5219 ts_tablet_manager.cc:1397] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.053s user 0.009s sys 0.003s
I20260927 23:28:53.986841 5219 raft_consensus.cc:348] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:53.989243 5221 tablet_bootstrap.cc:492] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:53.993645 5219 raft_consensus.cc:374] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:53.994596 5219 raft_consensus.cc:729] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:53.994868 5221 tablet_bootstrap.cc:654] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:53.996673 5219 consensus_queue.cc:260] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.006150 5219 ts_tablet_manager.cc:1428] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.023s user 0.002s sys 0.004s
I20260927 23:28:54.014096 5231 raft_consensus.cc:482] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:54.014566 5231 raft_consensus.cc:504] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.016065 5223 ts_tablet_manager.cc:1918] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:54.016569 5223 log.cc:1196] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/e7d138c6dbe5403d9fc7277bd9bb4fce
I20260927 23:28:54.017311 5223 ts_tablet_manager.cc:1939] T e7d138c6dbe5403d9fc7277bd9bb4fce P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:28:54.019976 5231 leader_election.cc:288] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:54.019971 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f8e7644db24be5a758b437a5f471c0" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:54.021777 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f8e7644db24be5a758b437a5f471c0" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:54.023595 4769 leader_election.cc:341] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:54.024322 4769 leader_election.cc:341] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:54.024703 4769 leader_election.cc:302] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:54.025707 5231 raft_consensus.cc:2726] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:54.052906 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 43f8e7644db24be5a758b437a5f471c0
I20260927 23:28:54.055481 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet 43f8e7644db24be5a758b437a5f471c0 with delete_type TABLET_DATA_DELETED (replaced by tablet cc208e8764244ac68b2a73be1b3d957e) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:54.055886 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 43f8e7644db24be5a758b437a5f471c0 with delete_type TABLET_DATA_DELETED (replaced by tablet cc208e8764244ac68b2a73be1b3d957e) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:54.057796 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 43f8e7644db24be5a758b437a5f471c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43f8e7644db24be5a758b437a5f471c0 already in progress: creating tablet
I20260927 23:28:54.058528 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet e7d138c6dbe5403d9fc7277bd9bb4fce (table default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:28:54.059247 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 43f8e7644db24be5a758b437a5f471c0 deleted and replaced by tablet cc208e8764244ac68b2a73be1b3d957e
W20260927 23:28:54.060781 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 14b3852ae71c48fc830ec6939a53c18a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 14b3852ae71c48fc830ec6939a53c18a already in progress: creating tablet
W20260927 23:28:54.062578 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 3a0c6ad197164aa98d73d357de26b488 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 3a0c6ad197164aa98d73d357de26b488 already in progress: creating tablet
I20260927 23:28:54.065292 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet e0f9998e6eb24f0a8fa217985ce3b07d from {username='slave'} at 127.0.0.1:35136
W20260927 23:28:54.068841 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 0a3148da66024c4684d5eaf3d60c8c58 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0a3148da66024c4684d5eaf3d60c8c58 already in progress: creating tablet
I20260927 23:28:54.069752 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 43f8e7644db24be5a758b437a5f471c0 with delete_type TABLET_DATA_DELETED (replaced by tablet cc208e8764244ac68b2a73be1b3d957e) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:54.070562 5224 tablet_replica.cc:330] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:54.071287 5224 raft_consensus.cc:2229] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:54.071751 5224 raft_consensus.cc:2258] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:54.068394 5229 tablet_bootstrap.cc:492] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:54.072422 5229 ts_tablet_manager.cc:1397] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.166s user 0.014s sys 0.000s
I20260927 23:28:54.074518 5224 ts_tablet_manager.cc:1905] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:54.084460 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 43f8e7644db24be5a758b437a5f471c0 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 43f8e7644db24be5a758b437a5f471c0 already in progress: creating tablet
I20260927 23:28:54.087663 4812 tablet_service.cc:1467] Processing CreateTablet for tablet ac88a4e4bee64586bef7b6b409a63f4c (DEFAULT_TABLE table=default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.089860 4954 tablet_service.cc:1467] Processing CreateTablet for tablet ac88a4e4bee64586bef7b6b409a63f4c (DEFAULT_TABLE table=default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.090219 5098 tablet_service.cc:1467] Processing CreateTablet for tablet ac88a4e4bee64586bef7b6b409a63f4c (DEFAULT_TABLE table=default.table_0275a380b3fe49989cf0c54688f42d05 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.091009 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac88a4e4bee64586bef7b6b409a63f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.091326 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac88a4e4bee64586bef7b6b409a63f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.086171 5221 tablet_bootstrap.cc:492] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:54.096760 5221 ts_tablet_manager.cc:1397] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.108s user 0.011s sys 0.001s
I20260927 23:28:54.100471 5221 raft_consensus.cc:348] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.101223 5221 raft_consensus.cc:374] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:54.101476 5221 raft_consensus.cc:729] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:54.102648 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac88a4e4bee64586bef7b6b409a63f4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.103037 5224 ts_tablet_manager.cc:1918] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:54.103467 5231 raft_consensus.cc:482] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:54.104051 5231 raft_consensus.cc:504] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.106062 5231 leader_election.cc:288] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:54.106490 5229 raft_consensus.cc:348] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.107236 5229 raft_consensus.cc:374] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:54.107487 5229 raft_consensus.cc:729] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:54.108104 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a73bf1fc35439ea3e39ffd843ebfb1" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:54.108253 5229 consensus_queue.cc:260] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:54.110458 4769 leader_election.cc:341] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:54.111282 5224 log.cc:1196] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/43f8e7644db24be5a758b437a5f471c0
I20260927 23:28:54.111090 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a73bf1fc35439ea3e39ffd843ebfb1" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:54.112527 5224 ts_tablet_manager.cc:1939] T 43f8e7644db24be5a758b437a5f471c0 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
W20260927 23:28:54.112861 4769 leader_election.cc:341] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:54.113866 4769 leader_election.cc:302] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:54.114692 5231 raft_consensus.cc:2726] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:54.118427 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 43f8e7644db24be5a758b437a5f471c0 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:54.145984 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:28:54.146183 5221 consensus_queue.cc:260] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.148703 5096 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f9998e6eb24f0a8fa217985ce3b07d with delete_type TABLET_DATA_DELETED (replaced by tablet abb39e8b5d2946ceae480a6a6f9cefb1) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:54.150671 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet e0f9998e6eb24f0a8fa217985ce3b07d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e0f9998e6eb24f0a8fa217985ce3b07d already in progress: creating tablet
I20260927 23:28:54.151062 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f9998e6eb24f0a8fa217985ce3b07d with delete_type TABLET_DATA_DELETED (replaced by tablet abb39e8b5d2946ceae480a6a6f9cefb1) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:54.152141 4622 catalog_manager.cc:6565] ReplaceTablet: tablet e0f9998e6eb24f0a8fa217985ce3b07d deleted and replaced by tablet abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:28:54.152840 5223 tablet_replica.cc:330] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:28:54.153236 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet e0f9998e6eb24f0a8fa217985ce3b07d with delete_type TABLET_DATA_DELETED (replaced by tablet abb39e8b5d2946ceae480a6a6f9cefb1) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:54.153734 5223 raft_consensus.cc:2229] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:54.153981 5224 tablet_replica.cc:330] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:54.154232 5223 raft_consensus.cc:2258] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:54.154584 5224 raft_consensus.cc:2229] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:54.155040 5224 raft_consensus.cc:2258] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:54.157035 5224 ts_tablet_manager.cc:1905] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:54.167526 5223 ts_tablet_manager.cc:1905] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:54.177574 5221 ts_tablet_manager.cc:1428] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.081s user 0.006s sys 0.001s
I20260927 23:28:54.199753 5224 ts_tablet_manager.cc:1918] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:54.203564 5224 log.cc:1196] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:28:54.205066 5224 ts_tablet_manager.cc:1939] T e0f9998e6eb24f0a8fa217985ce3b07d P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:54.207231 5229 ts_tablet_manager.cc:1428] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.135s user 0.007s sys 0.000s
I20260927 23:28:54.212400 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet e0f9998e6eb24f0a8fa217985ce3b07d (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
I20260927 23:28:54.220583 5223 ts_tablet_manager.cc:1918] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:54.221092 5223 log.cc:1196] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:28:54.221856 5223 ts_tablet_manager.cc:1939] T e0f9998e6eb24f0a8fa217985ce3b07d P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:28:54.224639 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet e0f9998e6eb24f0a8fa217985ce3b07d (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
I20260927 23:28:54.229992 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 14b3852ae71c48fc830ec6939a53c18a with delete_type TABLET_DATA_DELETED (replaced by tablet 43f8e7644db24be5a758b437a5f471c0) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:54.231815 5229 tablet_bootstrap.cc:492] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:54.232867 5223 tablet_replica.cc:330] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:28:54.233732 5223 raft_consensus.cc:2229] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:54.234268 5223 raft_consensus.cc:2258] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:54.234658 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:28:54.234964 5221 tablet_bootstrap.cc:492] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:54.245553 4687 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet ac88a4e4bee64586bef7b6b409a63f4c consensus state has no leader
I20260927 23:28:54.249703 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet e7d138c6dbe5403d9fc7277bd9bb4fce with delete_type TABLET_DATA_DELETED (replaced by tablet 0a3148da66024c4684d5eaf3d60c8c58) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:54.250723 5222 tablet_replica.cc:330] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a: stopping tablet replica
I20260927 23:28:54.251540 5222 raft_consensus.cc:2229] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:54.251893 5223 ts_tablet_manager.cc:1905] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:54.252048 5222 raft_consensus.cc:2258] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:54.259379 5222 ts_tablet_manager.cc:1905] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:54.263113 4624 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_0ec8d1f3b75e4c44b6b60deb61245f76 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:28:54.267634 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet e7a73bf1fc35439ea3e39ffd843ebfb1 from {username='slave'} at 127.0.0.1:35136
W20260927 23:28:54.267881 4623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_77e2daeb57dc4c1d9e46b4e05c32c6f2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:54.272354 4625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_14122b0b934a4107ab5120525f201e5c in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:28:54.277069 4615 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ac88a4e4bee64586bef7b6b409a63f4c consensus state has no leader
W20260927 23:28:54.278452 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet e7d138c6dbe5403d9fc7277bd9bb4fce
I20260927 23:28:54.280100 5221 tablet_bootstrap.cc:654] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:54.297153 5223 ts_tablet_manager.cc:1918] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:54.297597 5223 log.cc:1196] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/14b3852ae71c48fc830ec6939a53c18a
I20260927 23:28:54.298259 5223 ts_tablet_manager.cc:1939] T 14b3852ae71c48fc830ec6939a53c18a P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:28:54.305181 5229 tablet_bootstrap.cc:654] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:54.307554 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet 14b3852ae71c48fc830ec6939a53c18a (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:54.330092 5219 tablet_bootstrap.cc:492] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:54.336663 5219 tablet_bootstrap.cc:654] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:54.346321 5221 tablet_bootstrap.cc:492] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:54.346930 5221 ts_tablet_manager.cc:1397] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.122s user 0.009s sys 0.002s
I20260927 23:28:54.347913 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:28:54.352273 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet e7a73bf1fc35439ea3e39ffd843ebfb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 667f1147ecae4874854935982dd19e96) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:54.353101 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet e7a73bf1fc35439ea3e39ffd843ebfb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 667f1147ecae4874854935982dd19e96) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:54.353924 5224 tablet_replica.cc:330] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:54.354521 5224 raft_consensus.cc:2229] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:54.354888 5219 tablet_bootstrap.cc:492] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:54.354971 5224 raft_consensus.cc:2258] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:54.355445 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet e7a73bf1fc35439ea3e39ffd843ebfb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 667f1147ecae4874854935982dd19e96) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:54.357226 5224 ts_tablet_manager.cc:1905] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:54.360523 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet e7a73bf1fc35439ea3e39ffd843ebfb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7a73bf1fc35439ea3e39ffd843ebfb1 already in progress: creating tablet
I20260927 23:28:54.361831 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet e7a73bf1fc35439ea3e39ffd843ebfb1 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:54.349498 5221 raft_consensus.cc:348] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:54.364859 5221 raft_consensus.cc:374] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:54.365434 4622 catalog_manager.cc:6565] ReplaceTablet: tablet e7a73bf1fc35439ea3e39ffd843ebfb1 deleted and replaced by tablet 667f1147ecae4874854935982dd19e96
I20260927 23:28:54.355351 5219 ts_tablet_manager.cc:1397] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.005s
I20260927 23:28:54.367266 5221 raft_consensus.cc:729] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:54.370301 5098 tablet_service.cc:1467] Processing CreateTablet for tablet abb39e8b5d2946ceae480a6a6f9cefb1 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.371595 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb39e8b5d2946ceae480a6a6f9cefb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.374992 5219 raft_consensus.cc:348] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.377131 5219 raft_consensus.cc:374] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:54.377398 5219 raft_consensus.cc:729] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:54.378505 5221 consensus_queue.cc:260] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:54.383143 5224 ts_tablet_manager.cc:1918] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:54.373263 5222 ts_tablet_manager.cc:1918] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:54.383546 5224 log.cc:1196] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:28:54.384212 5222 log.cc:1196] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/e7d138c6dbe5403d9fc7277bd9bb4fce
W20260927 23:28:54.391620 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:54.394805 4812 tablet_service.cc:1467] Processing CreateTablet for tablet abb39e8b5d2946ceae480a6a6f9cefb1 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.395694 4954 tablet_service.cc:1467] Processing CreateTablet for tablet abb39e8b5d2946ceae480a6a6f9cefb1 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.396258 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb39e8b5d2946ceae480a6a6f9cefb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.396741 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet abb39e8b5d2946ceae480a6a6f9cefb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.400990 5219 consensus_queue.cc:260] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.405344 5221 ts_tablet_manager.cc:1428] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.058s user 0.006s sys 0.000s
W20260927 23:28:54.409844 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:54.413031 5222 ts_tablet_manager.cc:1939] T e7d138c6dbe5403d9fc7277bd9bb4fce P f5ec98b2972d44838fa273eb7a10d01a: Deleting consensus metadata
I20260927 23:28:54.415406 4622 catalog_manager.cc:5587] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571) does not have the latest schema for tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]). Expected version 1 got 0
I20260927 23:28:54.414778 5219 ts_tablet_manager.cc:1428] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.048s user 0.005s sys 0.000s
I20260927 23:28:54.419827 4813 tablet_service.cc:1467] Processing CreateTablet for tablet cc208e8764244ac68b2a73be1b3d957e (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.421558 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc208e8764244ac68b2a73be1b3d957e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.422825 4627 catalog_manager.cc:5587] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571) does not have the latest schema for tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]). Expected version 1 got 0
I20260927 23:28:54.426941 4953 tablet_service.cc:1467] Processing CreateTablet for tablet cc208e8764244ac68b2a73be1b3d957e (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.428151 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc208e8764244ac68b2a73be1b3d957e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.434106 5096 tablet_service.cc:1467] Processing CreateTablet for tablet cc208e8764244ac68b2a73be1b3d957e (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.384198 5224 ts_tablet_manager.cc:1939] T e7a73bf1fc35439ea3e39ffd843ebfb1 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:54.402886 5229 tablet_bootstrap.cc:492] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:54.442658 5229 ts_tablet_manager.cc:1397] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.211s user 0.011s sys 0.004s
I20260927 23:28:54.443578 5240 raft_consensus.cc:482] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:54.444059 5240 raft_consensus.cc:504] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:54.446112 5240 leader_election.cc:288] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
I20260927 23:28:54.448345 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c7f48abe7db4a5aa39892af9a3e3713" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:54.452224 4910 leader_election.cc:341] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:54.453222 4907 leader_election.cc:341] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:28:54.453662 4907 leader_election.cc:302] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:54.455011 5240 raft_consensus.cc:2726] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:54.458317 5229 raft_consensus.cc:348] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:54.459203 5229 raft_consensus.cc:374] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:54.459906 5229 raft_consensus.cc:729] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:54.462515 4629 catalog_manager.cc:3411] renamed table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863] in HMS: new name default.table_23bd70063a25487ab35a76f05dd4b81b
I20260927 23:28:54.462445 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet e7a73bf1fc35439ea3e39ffd843ebfb1 (table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]) successfully deleted
I20260927 23:28:54.478580 5219 tablet_bootstrap.cc:492] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:54.485405 4617 catalog_manager.cc:4841] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): tablet e7d138c6dbe5403d9fc7277bd9bb4fce (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
W20260927 23:28:54.487627 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet e7a73bf1fc35439ea3e39ffd843ebfb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e7a73bf1fc35439ea3e39ffd843ebfb1 already in progress: creating tablet
I20260927 23:28:54.488942 5221 tablet_bootstrap.cc:492] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:54.498733 5219 tablet_bootstrap.cc:654] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:54.480741 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc208e8764244ac68b2a73be1b3d957e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.510145 4813 tablet_service.cc:1467] Processing CreateTablet for tablet 667f1147ecae4874854935982dd19e96 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.511426 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667f1147ecae4874854935982dd19e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.514329 5240 raft_consensus.cc:482] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:54.514885 5240 raft_consensus.cc:504] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:54.518545 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0c6ad197164aa98d73d357de26b488" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:54.521116 5240 leader_election.cc:288] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:28:54.523998 4910 leader_election.cc:341] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:54.525842 4907 leader_election.cc:341] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:54.526492 4907 leader_election.cc:302] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:54.529762 5240 raft_consensus.cc:2726] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:54.529902 5098 tablet_service.cc:1467] Processing CreateTablet for tablet 667f1147ecae4874854935982dd19e96 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.530277 4954 tablet_service.cc:1467] Processing CreateTablet for tablet 667f1147ecae4874854935982dd19e96 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.531230 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667f1147ecae4874854935982dd19e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.531327 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 667f1147ecae4874854935982dd19e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.536988 5229 consensus_queue.cc:260] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:54.539796 5229 ts_tablet_manager.cc:1428] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.097s user 0.006s sys 0.000s
I20260927 23:28:54.542906 5219 tablet_bootstrap.cc:492] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:54.543398 5219 ts_tablet_manager.cc:1397] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.065s user 0.009s sys 0.004s
I20260927 23:28:54.546550 5219 raft_consensus.cc:348] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.547267 5219 raft_consensus.cc:374] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:54.547528 5219 raft_consensus.cc:729] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:54.559666 5221 tablet_bootstrap.cc:654] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:54.548172 5219 consensus_queue.cc:260] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.573578 5229 tablet_bootstrap.cc:492] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:54.579834 5219 ts_tablet_manager.cc:1428] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.035s user 0.008s sys 0.000s
I20260927 23:28:54.583194 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet cc208e8764244ac68b2a73be1b3d957e from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:54.584345 5219 tablet_bootstrap.cc:492] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
W20260927 23:28:54.590420 4616 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.62:35155, user_credentials={real_user=slave}} blocked reactor thread for 116730us
W20260927 23:28:54.593623 4616 consensus_peers.cc:487] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d -> Peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Couldn't send request to peer d9ed71d97661411891f2d2de4cdc2269. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260927 23:28:54.595031 5219 tablet_bootstrap.cc:654] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:54.602977 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:54.604671 4622 catalog_manager.cc:6565] ReplaceTablet: tablet cc208e8764244ac68b2a73be1b3d957e deleted and replaced by tablet a483b5e32d55406bb3342af93f755028
W20260927 23:28:54.606638 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 9912e4dd70ba482398fbd831a8242f12
I20260927 23:28:54.606837 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet cc208e8764244ac68b2a73be1b3d957e with delete_type TABLET_DATA_DELETED (replaced by tablet a483b5e32d55406bb3342af93f755028) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:54.607573 5224 tablet_replica.cc:330] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:54.608104 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet cc208e8764244ac68b2a73be1b3d957e with delete_type TABLET_DATA_DELETED (replaced by tablet a483b5e32d55406bb3342af93f755028) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:54.608263 5224 raft_consensus.cc:2229] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:54.608743 5224 raft_consensus.cc:2258] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:54.610127 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet cc208e8764244ac68b2a73be1b3d957e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet cc208e8764244ac68b2a73be1b3d957e already in progress: creating tablet
I20260927 23:28:54.611063 5224 ts_tablet_manager.cc:1905] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:54.611677 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet ac88a4e4bee64586bef7b6b409a63f4c from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:54.621023 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet cc208e8764244ac68b2a73be1b3d957e with delete_type TABLET_DATA_DELETED (replaced by tablet a483b5e32d55406bb3342af93f755028) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:54.624547 4617 catalog_manager.cc:4808] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet cc208e8764244ac68b2a73be1b3d957e because the tablet was not found. No further retry: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:54.643313 5224 ts_tablet_manager.cc:1918] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:54.643760 5224 log.cc:1196] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:54.644515 5224 ts_tablet_manager.cc:1939] T cc208e8764244ac68b2a73be1b3d957e P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:54.647120 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet cc208e8764244ac68b2a73be1b3d957e (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:54.654395 5221 tablet_bootstrap.cc:492] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:54.655016 5221 ts_tablet_manager.cc:1397] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.166s user 0.010s sys 0.002s
I20260927 23:28:54.656957 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet cc208e8764244ac68b2a73be1b3d957e with delete_type TABLET_DATA_DELETED (replaced by tablet a483b5e32d55406bb3342af93f755028) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:54.657719 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet ac88a4e4bee64586bef7b6b409a63f4c
I20260927 23:28:54.657755 5221 raft_consensus.cc:348] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:54.658605 5221 raft_consensus.cc:374] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260927 23:28:54.658557 4614 catalog_manager.cc:4808] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet cc208e8764244ac68b2a73be1b3d957e because the tablet was not found. No further retry: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:54.658915 5221 raft_consensus.cc:729] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:54.659895 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet ac88a4e4bee64586bef7b6b409a63f4c with delete_type TABLET_DATA_DELETED (replaced by tablet 38cd408c3dc34fabbab5906286e7580a) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:54.659997 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet ac88a4e4bee64586bef7b6b409a63f4c with delete_type TABLET_DATA_DELETED (replaced by tablet 38cd408c3dc34fabbab5906286e7580a) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:54.660312 5229 tablet_bootstrap.cc:654] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:54.660765 5224 tablet_replica.cc:330] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:54.661391 5224 raft_consensus.cc:2229] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
W20260927 23:28:54.661753 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet ac88a4e4bee64586bef7b6b409a63f4c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac88a4e4bee64586bef7b6b409a63f4c already in progress: creating tablet
I20260927 23:28:54.661891 5224 raft_consensus.cc:2258] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:54.667538 5224 ts_tablet_manager.cc:1905] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:54.671583 4627 catalog_manager.cc:6565] ReplaceTablet: tablet ac88a4e4bee64586bef7b6b409a63f4c deleted and replaced by tablet 38cd408c3dc34fabbab5906286e7580a
I20260927 23:28:54.671677 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet ac88a4e4bee64586bef7b6b409a63f4c with delete_type TABLET_DATA_DELETED (replaced by tablet 38cd408c3dc34fabbab5906286e7580a) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:54.673398 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet ac88a4e4bee64586bef7b6b409a63f4c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ac88a4e4bee64586bef7b6b409a63f4c already in progress: creating tablet
I20260927 23:28:54.675423 5219 tablet_bootstrap.cc:492] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:54.675918 5219 ts_tablet_manager.cc:1397] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.095s user 0.007s sys 0.020s
I20260927 23:28:54.659663 5221 consensus_queue.cc:260] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
W20260927 23:28:54.677994 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 9912e4dd70ba482398fbd831a8242f12
I20260927 23:28:54.681259 5219 raft_consensus.cc:348] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.682006 5219 raft_consensus.cc:374] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:54.682265 5219 raft_consensus.cc:729] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:54.690600 5221 ts_tablet_manager.cc:1428] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.035s user 0.005s sys 0.000s
I20260927 23:28:54.682884 5219 consensus_queue.cc:260] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.699167 5219 ts_tablet_manager.cc:1428] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.023s user 0.002s sys 0.005s
I20260927 23:28:54.703439 5219 tablet_bootstrap.cc:492] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
W20260927 23:28:54.704821 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
I20260927 23:28:54.718644 5219 tablet_bootstrap.cc:654] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:54.718578 5229 tablet_bootstrap.cc:492] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:54.697592 5221 tablet_bootstrap.cc:492] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:54.719332 5229 ts_tablet_manager.cc:1397] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.146s user 0.012s sys 0.000s
I20260927 23:28:54.722280 5229 raft_consensus.cc:348] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:54.723361 5229 raft_consensus.cc:374] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:54.723944 5229 raft_consensus.cc:729] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:54.725564 5224 ts_tablet_manager.cc:1918] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:54.725962 5224 log.cc:1196] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/ac88a4e4bee64586bef7b6b409a63f4c
I20260927 23:28:54.726662 5224 ts_tablet_manager.cc:1939] T ac88a4e4bee64586bef7b6b409a63f4c P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
W20260927 23:28:54.727408 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
I20260927 23:28:54.731968 5219 tablet_bootstrap.cc:492] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:54.732445 5219 ts_tablet_manager.cc:1397] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.032s user 0.010s sys 0.000s
I20260927 23:28:54.734825 5219 raft_consensus.cc:348] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:54.735520 5219 raft_consensus.cc:374] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:54.735754 5219 raft_consensus.cc:729] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:54.740222 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:28:54.743522 5219 consensus_queue.cc:260] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:54.749915 5219 ts_tablet_manager.cc:1428] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.017s user 0.005s sys 0.000s
I20260927 23:28:54.750303 5221 tablet_bootstrap.cc:654] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:54.725328 5229 consensus_queue.cc:260] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
W20260927 23:28:54.777869 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet f171cc1f0094480a9cc6357eab93dff1
I20260927 23:28:54.783080 5229 ts_tablet_manager.cc:1428] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.063s user 0.006s sys 0.000s
I20260927 23:28:54.786381 4812 tablet_service.cc:1467] Processing CreateTablet for tablet a483b5e32d55406bb3342af93f755028 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.787631 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a483b5e32d55406bb3342af93f755028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.790068 4813 tablet_service.cc:1467] Processing CreateTablet for tablet 38cd408c3dc34fabbab5906286e7580a (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.791210 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38cd408c3dc34fabbab5906286e7580a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.791086 4953 tablet_service.cc:1467] Processing CreateTablet for tablet a483b5e32d55406bb3342af93f755028 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.792209 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a483b5e32d55406bb3342af93f755028. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:54.796201 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet f171cc1f0094480a9cc6357eab93dff1
I20260927 23:28:54.797164 5098 tablet_service.cc:1467] Processing CreateTablet for tablet a483b5e32d55406bb3342af93f755028 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.798760 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a483b5e32d55406bb3342af93f755028. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.798852 5229 tablet_bootstrap.cc:492] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:54.808140 4954 tablet_service.cc:1467] Processing CreateTablet for tablet 38cd408c3dc34fabbab5906286e7580a (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.809291 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38cd408c3dc34fabbab5906286e7580a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.811898 5219 tablet_bootstrap.cc:492] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:54.814520 5096 tablet_service.cc:1467] Processing CreateTablet for tablet 38cd408c3dc34fabbab5906286e7580a (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:54.815659 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 38cd408c3dc34fabbab5906286e7580a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.821826 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 667f1147ecae4874854935982dd19e96 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:54.851531 5240 raft_consensus.cc:482] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:54.852006 5240 raft_consensus.cc:504] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:54.853394 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 667f1147ecae4874854935982dd19e96
I20260927 23:28:54.853950 5240 leader_election.cc:288] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
I20260927 23:28:54.855630 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3148da66024c4684d5eaf3d60c8c58" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:54.857378 4910 leader_election.cc:341] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:54.860889 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet 667f1147ecae4874854935982dd19e96 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dcb78350f4c4b73beada474bcb01169) from {username='slave'} at 127.0.0.1:34620
W20260927 23:28:54.862823 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 667f1147ecae4874854935982dd19e96 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 667f1147ecae4874854935982dd19e96 already in progress: creating tablet
I20260927 23:28:54.865267 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet 667f1147ecae4874854935982dd19e96 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dcb78350f4c4b73beada474bcb01169) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:54.867051 5229 tablet_bootstrap.cc:654] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:54.867985 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 667f1147ecae4874854935982dd19e96 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 667f1147ecae4874854935982dd19e96 already in progress: creating tablet
I20260927 23:28:54.872783 4622 catalog_manager.cc:6565] ReplaceTablet: tablet 667f1147ecae4874854935982dd19e96 deleted and replaced by tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:28:54.882277 5221 tablet_bootstrap.cc:492] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:54.882846 5221 ts_tablet_manager.cc:1397] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.185s user 0.009s sys 0.003s
I20260927 23:28:54.886668 5221 raft_consensus.cc:348] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.887458 5221 raft_consensus.cc:374] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:54.887763 5221 raft_consensus.cc:729] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:54.888935 5221 consensus_queue.cc:260] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:54.899122 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet 667f1147ecae4874854935982dd19e96 with delete_type TABLET_DATA_DELETED (replaced by tablet 2dcb78350f4c4b73beada474bcb01169) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:54.899859 5224 tablet_replica.cc:330] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:54.900501 5224 raft_consensus.cc:2229] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:54.900938 5224 raft_consensus.cc:2258] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
W20260927 23:28:54.902273 4907 leader_election.cc:341] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58
I20260927 23:28:54.902709 4907 leader_election.cc:302] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:54.903121 5224 ts_tablet_manager.cc:1905] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:54.906172 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 38cd408c3dc34fabbab5906286e7580a from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:54.913008 5229 tablet_bootstrap.cc:492] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:54.913472 5229 ts_tablet_manager.cc:1397] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.115s user 0.010s sys 0.001s
I20260927 23:28:54.919749 5224 ts_tablet_manager.cc:1918] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:54.920125 5224 log.cc:1196] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/667f1147ecae4874854935982dd19e96
I20260927 23:28:54.928339 5224 ts_tablet_manager.cc:1939] T 667f1147ecae4874854935982dd19e96 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:54.945717 5219 tablet_bootstrap.cc:654] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:54.952315 5240 raft_consensus.cc:2726] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:54.967083 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
I20260927 23:28:54.981729 5221 ts_tablet_manager.cc:1428] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.099s user 0.002s sys 0.003s
I20260927 23:28:54.987032 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:28:54.992965 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
I20260927 23:28:54.993443 5096 tablet_service.cc:1514] Processing DeleteTablet for tablet 38cd408c3dc34fabbab5906286e7580a with delete_type TABLET_DATA_DELETED (replaced by tablet 360070544fca4f62b0aac16d5889e1bc) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:54.994303 4622 catalog_manager.cc:6565] ReplaceTablet: tablet 38cd408c3dc34fabbab5906286e7580a deleted and replaced by tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:28:54.995669 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 38cd408c3dc34fabbab5906286e7580a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 38cd408c3dc34fabbab5906286e7580a already in progress: creating tablet
I20260927 23:28:55.005626 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 667f1147ecae4874854935982dd19e96 (table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]) successfully deleted
I20260927 23:28:55.008601 4811 tablet_service.cc:1467] Processing CreateTablet for tablet 2dcb78350f4c4b73beada474bcb01169 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.010344 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dcb78350f4c4b73beada474bcb01169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.010802 5096 tablet_service.cc:1467] Processing CreateTablet for tablet 2dcb78350f4c4b73beada474bcb01169 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.011966 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dcb78350f4c4b73beada474bcb01169. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:55.013548 4907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:40571, user_credentials={real_user=slave}} blocked reactor thread for 111256us
I20260927 23:28:55.017627 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet 38cd408c3dc34fabbab5906286e7580a with delete_type TABLET_DATA_DELETED (replaced by tablet 360070544fca4f62b0aac16d5889e1bc) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:55.024766 4954 tablet_service.cc:1467] Processing CreateTablet for tablet 2dcb78350f4c4b73beada474bcb01169 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.026090 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dcb78350f4c4b73beada474bcb01169. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:54.916008 5229 raft_consensus.cc:348] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:55.027544 5229 raft_consensus.cc:374] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:55.027850 5229 raft_consensus.cc:729] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
W20260927 23:28:55.028710 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet 38cd408c3dc34fabbab5906286e7580a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 38cd408c3dc34fabbab5906286e7580a already in progress: creating tablet
I20260927 23:28:55.031948 5221 tablet_bootstrap.cc:492] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:55.035046 5219 tablet_bootstrap.cc:492] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:55.035678 5219 ts_tablet_manager.cc:1397] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.224s user 0.014s sys 0.001s
I20260927 23:28:55.038534 5219 raft_consensus.cc:348] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.037187 5229 consensus_queue.cc:260] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:55.040171 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 38cd408c3dc34fabbab5906286e7580a with delete_type TABLET_DATA_DELETED (replaced by tablet 360070544fca4f62b0aac16d5889e1bc) from {username='slave'} at 127.0.0.1:34620
W20260927 23:28:55.042109 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 38cd408c3dc34fabbab5906286e7580a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 38cd408c3dc34fabbab5906286e7580a already in progress: creating tablet
I20260927 23:28:55.056061 5219 raft_consensus.cc:374] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260927 23:28:55.057768 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:28:55.061055 5229 ts_tablet_manager.cc:1428] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.147s user 0.005s sys 0.002s
I20260927 23:28:55.061455 5219 raft_consensus.cc:729] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:55.062356 5219 consensus_queue.cc:260] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.065393 5221 tablet_bootstrap.cc:654] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:55.075729 5219 ts_tablet_manager.cc:1428] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.040s user 0.006s sys 0.000s
I20260927 23:28:55.088958 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet a483b5e32d55406bb3342af93f755028 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:55.091614 5219 tablet_bootstrap.cc:492] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:55.104545 5219 tablet_bootstrap.cc:654] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:55.114746 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet a483b5e32d55406bb3342af93f755028
I20260927 23:28:55.116421 4627 catalog_manager.cc:6565] ReplaceTablet: tablet a483b5e32d55406bb3342af93f755028 deleted and replaced by tablet a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:28:55.116959 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet a483b5e32d55406bb3342af93f755028 with delete_type TABLET_DATA_DELETED (replaced by tablet a007bdab1e3c4e40a259b97f1e43bd51) from {username='slave'} at 127.0.0.1:53202
W20260927 23:28:55.119024 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet a483b5e32d55406bb3342af93f755028 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a483b5e32d55406bb3342af93f755028 already in progress: creating tablet
I20260927 23:28:55.119925 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet a483b5e32d55406bb3342af93f755028 with delete_type TABLET_DATA_DELETED (replaced by tablet a007bdab1e3c4e40a259b97f1e43bd51) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:55.123174 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet a483b5e32d55406bb3342af93f755028 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a483b5e32d55406bb3342af93f755028 already in progress: creating tablet
I20260927 23:28:55.124171 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet a483b5e32d55406bb3342af93f755028 with delete_type TABLET_DATA_DELETED (replaced by tablet a007bdab1e3c4e40a259b97f1e43bd51) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:55.136124 5229 tablet_bootstrap.cc:492] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
W20260927 23:28:55.142140 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:28:55.143046 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet a483b5e32d55406bb3342af93f755028 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a483b5e32d55406bb3342af93f755028 already in progress: creating tablet
I20260927 23:28:55.143203 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 38cd408c3dc34fabbab5906286e7580a with delete_type TABLET_DATA_DELETED (replaced by tablet 360070544fca4f62b0aac16d5889e1bc) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:55.144176 5224 tablet_replica.cc:330] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:55.144831 5224 raft_consensus.cc:2229] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:55.145247 5224 raft_consensus.cc:2258] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:55.147488 5224 ts_tablet_manager.cc:1905] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:55.152868 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
I20260927 23:28:55.166833 5221 tablet_bootstrap.cc:492] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:55.167371 5221 ts_tablet_manager.cc:1397] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.162s user 0.011s sys 0.000s
I20260927 23:28:55.170599 5221 raft_consensus.cc:348] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.171360 5221 raft_consensus.cc:374] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:55.171610 5221 raft_consensus.cc:729] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:55.172067 5231 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:55.172381 5221 consensus_queue.cc:260] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.172775 5231 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:55.175158 5231 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:55.172017 5224 ts_tablet_manager.cc:1918] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:55.177114 5229 tablet_bootstrap.cc:654] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:55.177147 5224 log.cc:1196] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/38cd408c3dc34fabbab5906286e7580a
I20260927 23:28:55.178002 5224 ts_tablet_manager.cc:1939] T 38cd408c3dc34fabbab5906286e7580a P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:55.178225 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb39e8b5d2946ceae480a6a6f9cefb1" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:55.179541 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "abb39e8b5d2946ceae480a6a6f9cefb1" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:55.181427 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 38cd408c3dc34fabbab5906286e7580a (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
W20260927 23:28:55.183323 4769 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:55.184015 4769 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:55.184401 4769 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:55.185786 5231 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:55.188279 5219 tablet_bootstrap.cc:492] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:55.188936 5219 ts_tablet_manager.cc:1397] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.098s user 0.012s sys 0.006s
I20260927 23:28:55.191720 5219 raft_consensus.cc:348] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.200982 5219 raft_consensus.cc:374] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:55.201542 5219 raft_consensus.cc:729] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:55.204398 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet abb39e8b5d2946ceae480a6a6f9cefb1 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:55.204671 4953 tablet_service.cc:1467] Processing CreateTablet for tablet a007bdab1e3c4e40a259b97f1e43bd51 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.205925 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a007bdab1e3c4e40a259b97f1e43bd51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.211202 5098 tablet_service.cc:1467] Processing CreateTablet for tablet a007bdab1e3c4e40a259b97f1e43bd51 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.211148 4812 tablet_service.cc:1467] Processing CreateTablet for tablet 360070544fca4f62b0aac16d5889e1bc (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.212327 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360070544fca4f62b0aac16d5889e1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.213234 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a007bdab1e3c4e40a259b97f1e43bd51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.217217 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 360070544fca4f62b0aac16d5889e1bc (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.218387 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360070544fca4f62b0aac16d5889e1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.220881 4952 tablet_service.cc:1467] Processing CreateTablet for tablet 360070544fca4f62b0aac16d5889e1bc (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.222046 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 360070544fca4f62b0aac16d5889e1bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.231596 5219 consensus_queue.cc:260] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.234522 5229 tablet_bootstrap.cc:492] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:55.235011 5229 ts_tablet_manager.cc:1397] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.099s user 0.015s sys 0.033s
I20260927 23:28:55.249480 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:28:55.250432 5219 ts_tablet_manager.cc:1428] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.061s user 0.006s sys 0.000s
I20260927 23:28:55.251500 5095 tablet_service.cc:1514] Processing DeleteTablet for tablet abb39e8b5d2946ceae480a6a6f9cefb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 7a2477e4ce30497fa94f2dfdb3c31a3d) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:55.252279 5229 raft_consensus.cc:348] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:55.253008 5229 raft_consensus.cc:374] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:55.253315 5229 raft_consensus.cc:729] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
W20260927 23:28:55.253820 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet abb39e8b5d2946ceae480a6a6f9cefb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abb39e8b5d2946ceae480a6a6f9cefb1 already in progress: creating tablet
I20260927 23:28:55.256760 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet a483b5e32d55406bb3342af93f755028 with delete_type TABLET_DATA_DELETED (replaced by tablet a007bdab1e3c4e40a259b97f1e43bd51) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:55.257508 5224 tablet_replica.cc:330] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:55.258194 5224 raft_consensus.cc:2229] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:55.258333 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet abb39e8b5d2946ceae480a6a6f9cefb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 7a2477e4ce30497fa94f2dfdb3c31a3d) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:55.258673 5224 raft_consensus.cc:2258] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:55.259387 4951 tablet_service.cc:1514] Processing DeleteTablet for tablet abb39e8b5d2946ceae480a6a6f9cefb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 7a2477e4ce30497fa94f2dfdb3c31a3d) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:55.260370 4627 catalog_manager.cc:6565] ReplaceTablet: tablet abb39e8b5d2946ceae480a6a6f9cefb1 deleted and replaced by tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
I20260927 23:28:55.263447 5224 ts_tablet_manager.cc:1905] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:55.268198 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet abb39e8b5d2946ceae480a6a6f9cefb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abb39e8b5d2946ceae480a6a6f9cefb1 already in progress: creating tablet
I20260927 23:28:55.275740 5219 tablet_bootstrap.cc:492] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:55.276477 4811 tablet_service.cc:1467] Processing CreateTablet for tablet a007bdab1e3c4e40a259b97f1e43bd51 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.277642 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a007bdab1e3c4e40a259b97f1e43bd51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.279398 5229 consensus_queue.cc:260] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:55.283916 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 7a2477e4ce30497fa94f2dfdb3c31a3d (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.285216 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a2477e4ce30497fa94f2dfdb3c31a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.291790 5229 ts_tablet_manager.cc:1428] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.056s user 0.008s sys 0.000s
I20260927 23:28:55.292220 4951 tablet_service.cc:1467] Processing CreateTablet for tablet 7a2477e4ce30497fa94f2dfdb3c31a3d (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.292999 5229 tablet_bootstrap.cc:492] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:55.293398 4951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a2477e4ce30497fa94f2dfdb3c31a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.297467 5219 tablet_bootstrap.cc:654] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:55.300474 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:28:55.305226 5095 tablet_service.cc:1467] Processing CreateTablet for tablet 7a2477e4ce30497fa94f2dfdb3c31a3d (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.310007 5095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a2477e4ce30497fa94f2dfdb3c31a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.315296 5229 tablet_bootstrap.cc:654] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:55.325290 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 2dcb78350f4c4b73beada474bcb01169 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:55.325785 5219 tablet_bootstrap.cc:492] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:55.329367 5219 ts_tablet_manager.cc:1397] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.068s user 0.011s sys 0.001s
I20260927 23:28:55.331940 5219 raft_consensus.cc:348] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:55.336017 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:28:55.338474 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:28:55.339450 5296 raft_consensus.cc:482] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:55.339936 5296 raft_consensus.cc:504] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.342079 5296 leader_election.cc:288] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
W20260927 23:28:55.345330 4907 leader_election.cc:341] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0
I20260927 23:28:55.345546 5219 raft_consensus.cc:374] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:55.345865 5219 raft_consensus.cc:729] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:55.346916 5219 consensus_queue.cc:260] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.347458 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f8e7644db24be5a758b437a5f471c0" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:55.349009 4910 leader_election.cc:341] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:55.349422 4910 leader_election.cc:302] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:55.351154 5296 raft_consensus.cc:2726] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:55.362869 5219 ts_tablet_manager.cc:1428] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.033s user 0.005s sys 0.000s
I20260927 23:28:55.376248 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:28:55.378196 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 2dcb78350f4c4b73beada474bcb01169 deleted and replaced by tablet c5552715138046279f00dcf497d05a72
I20260927 23:28:55.380478 4813 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dcb78350f4c4b73beada474bcb01169 with delete_type TABLET_DATA_DELETED (replaced by tablet c5552715138046279f00dcf497d05a72) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:55.381196 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dcb78350f4c4b73beada474bcb01169 with delete_type TABLET_DATA_DELETED (replaced by tablet c5552715138046279f00dcf497d05a72) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:55.383776 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dcb78350f4c4b73beada474bcb01169 with delete_type TABLET_DATA_DELETED (replaced by tablet c5552715138046279f00dcf497d05a72) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:55.384548 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 2dcb78350f4c4b73beada474bcb01169 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2dcb78350f4c4b73beada474bcb01169 already in progress: creating tablet
W20260927 23:28:55.386251 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 2dcb78350f4c4b73beada474bcb01169 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2dcb78350f4c4b73beada474bcb01169 already in progress: creating tablet
I20260927 23:28:55.387125 5221 ts_tablet_manager.cc:1428] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.219s user 0.007s sys 0.000s
I20260927 23:28:55.398816 5221 tablet_bootstrap.cc:492] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:55.413461 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:28:55.424935 5219 tablet_bootstrap.cc:492] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:55.430855 5219 tablet_bootstrap.cc:654] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:55.451481 5224 ts_tablet_manager.cc:1918] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:55.451969 5224 log.cc:1196] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/a483b5e32d55406bb3342af93f755028
I20260927 23:28:55.452695 5224 ts_tablet_manager.cc:1939] T a483b5e32d55406bb3342af93f755028 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:55.455341 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet a483b5e32d55406bb3342af93f755028 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:55.460142 5221 tablet_bootstrap.cc:654] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:55.466693 5277 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:55.467301 5277 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:55.469523 5277 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
I20260927 23:28:55.469367 5224 tablet_replica.cc:330] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
W20260927 23:28:55.473258 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:28:55.474408 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:28:55.474148 5224 raft_consensus.cc:2229] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:55.474845 5054 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:28:55.475288 5224 raft_consensus.cc:2258] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:55.478995 5224 ts_tablet_manager.cc:1905] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:55.479105 4615 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ac88a4e4bee64586bef7b6b409a63f4c consensus state has no leader
I20260927 23:28:55.480057 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 360070544fca4f62b0aac16d5889e1bc from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:55.481197 5277 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:55.485023 4952 tablet_service.cc:1467] Processing CreateTablet for tablet c5552715138046279f00dcf497d05a72 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.486464 5095 tablet_service.cc:1467] Processing CreateTablet for tablet c5552715138046279f00dcf497d05a72 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.487672 5095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5552715138046279f00dcf497d05a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.496121 5229 tablet_bootstrap.cc:492] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:55.496730 5229 ts_tablet_manager.cc:1397] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.204s user 0.012s sys 0.021s
I20260927 23:28:55.501822 5219 tablet_bootstrap.cc:492] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:55.502065 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 360070544fca4f62b0aac16d5889e1bc
I20260927 23:28:55.502305 5219 ts_tablet_manager.cc:1397] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.078s user 0.005s sys 0.005s
I20260927 23:28:55.503906 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 360070544fca4f62b0aac16d5889e1bc with delete_type TABLET_DATA_DELETED (replaced by tablet 260ab7817da1431ebd80ee7dd00c40b4) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:55.505693 4622 catalog_manager.cc:6565] ReplaceTablet: tablet 360070544fca4f62b0aac16d5889e1bc deleted and replaced by tablet 260ab7817da1431ebd80ee7dd00c40b4
I20260927 23:28:55.506611 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5552715138046279f00dcf497d05a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.503999 5229 raft_consensus.cc:348] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.507229 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet 360070544fca4f62b0aac16d5889e1bc with delete_type TABLET_DATA_DELETED (replaced by tablet 260ab7817da1431ebd80ee7dd00c40b4) from {username='slave'} at 127.0.0.1:34620
W20260927 23:28:55.509596 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 360070544fca4f62b0aac16d5889e1bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 360070544fca4f62b0aac16d5889e1bc already in progress: creating tablet
I20260927 23:28:55.511366 5219 raft_consensus.cc:348] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.508620 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet 360070544fca4f62b0aac16d5889e1bc with delete_type TABLET_DATA_DELETED (replaced by tablet 260ab7817da1431ebd80ee7dd00c40b4) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:55.512135 5219 raft_consensus.cc:374] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:55.512418 5219 raft_consensus.cc:729] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:55.512791 4810 tablet_service.cc:1467] Processing CreateTablet for tablet c5552715138046279f00dcf497d05a72 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.513990 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c5552715138046279f00dcf497d05a72. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:55.513882 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 360070544fca4f62b0aac16d5889e1bc because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 360070544fca4f62b0aac16d5889e1bc already in progress: creating tablet
I20260927 23:28:55.514569 5229 raft_consensus.cc:374] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:55.514887 5229 raft_consensus.cc:729] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:55.516508 5229 consensus_queue.cc:260] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.518658 5221 tablet_bootstrap.cc:492] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:55.519240 5221 ts_tablet_manager.cc:1397] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.123s user 0.012s sys 0.000s
W20260927 23:28:55.534366 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:28:55.533675 5221 raft_consensus.cc:348] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.537147 5221 raft_consensus.cc:374] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:55.537415 5221 raft_consensus.cc:729] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:55.538160 5221 consensus_queue.cc:260] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.546178 5221 ts_tablet_manager.cc:1428] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
W20260927 23:28:55.557559 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
I20260927 23:28:55.558570 5229 ts_tablet_manager.cc:1428] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.061s user 0.006s sys 0.000s
I20260927 23:28:55.565776 5224 ts_tablet_manager.cc:1918] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:55.566202 5224 log.cc:1196] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:28:55.566866 5224 ts_tablet_manager.cc:1939] T abb39e8b5d2946ceae480a6a6f9cefb1 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:55.518505 5219 consensus_queue.cc:260] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:55.574275 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
I20260927 23:28:55.580796 5219 ts_tablet_manager.cc:1428] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.078s user 0.008s sys 0.000s
I20260927 23:28:55.586663 5219 tablet_bootstrap.cc:492] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:55.593954 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet abb39e8b5d2946ceae480a6a6f9cefb1 (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
I20260927 23:28:55.595378 5224 tablet_replica.cc:330] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:55.596258 5224 raft_consensus.cc:2229] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:55.596827 5224 raft_consensus.cc:2258] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:55.600266 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 260ab7817da1431ebd80ee7dd00c40b4 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.600868 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 7a2477e4ce30497fa94f2dfdb3c31a3d from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:55.601852 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 260ab7817da1431ebd80ee7dd00c40b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.609223 5224 ts_tablet_manager.cc:1905] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:55.547214 5221 tablet_bootstrap.cc:492] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:55.620088 4952 tablet_service.cc:1467] Processing CreateTablet for tablet 260ab7817da1431ebd80ee7dd00c40b4 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.621416 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 260ab7817da1431ebd80ee7dd00c40b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.630206 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 260ab7817da1431ebd80ee7dd00c40b4 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.630744 5229 tablet_bootstrap.cc:492] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:55.631397 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 260ab7817da1431ebd80ee7dd00c40b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.634361 5219 tablet_bootstrap.cc:654] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:55.652782 5221 tablet_bootstrap.cc:654] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:55.658596 5229 tablet_bootstrap.cc:654] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:55.661892 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:28:55.681977 5219 tablet_bootstrap.cc:492] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:55.682464 5219 ts_tablet_manager.cc:1397] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.096s user 0.007s sys 0.004s
I20260927 23:28:55.684315 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
I20260927 23:28:55.684688 5277 raft_consensus.cc:482] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:55.685191 5277 raft_consensus.cc:504] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:55.686283 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a2477e4ce30497fa94f2dfdb3c31a3d with delete_type TABLET_DATA_DELETED (replaced by tablet 2826716e5b184cc2b60d556ca2b89f4d) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:55.686969 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 7a2477e4ce30497fa94f2dfdb3c31a3d deleted and replaced by tablet 2826716e5b184cc2b60d556ca2b89f4d
I20260927 23:28:55.687187 5277 leader_election.cc:288] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:55.688741 4951 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a2477e4ce30497fa94f2dfdb3c31a3d with delete_type TABLET_DATA_DELETED (replaced by tablet 2826716e5b184cc2b60d556ca2b89f4d) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:55.689538 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a2477e4ce30497fa94f2dfdb3c31a3d with delete_type TABLET_DATA_DELETED (replaced by tablet 2826716e5b184cc2b60d556ca2b89f4d) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:55.689595 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c7f48abe7db4a5aa39892af9a3e3713" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:55.690413 4974 raft_consensus.cc:2454] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
W20260927 23:28:55.691399 5051 leader_election.cc:341] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:28:55.693569 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 7a2477e4ce30497fa94f2dfdb3c31a3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a2477e4ce30497fa94f2dfdb3c31a3d already in progress: creating tablet
I20260927 23:28:55.695122 5054 leader_election.cc:302] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
W20260927 23:28:55.696394 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 7a2477e4ce30497fa94f2dfdb3c31a3d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 7a2477e4ce30497fa94f2dfdb3c31a3d already in progress: creating tablet
I20260927 23:28:55.697248 5277 raft_consensus.cc:2781] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:55.697585 5277 raft_consensus.cc:482] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:55.697888 5277 raft_consensus.cc:3037] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:55.706161 5277 raft_consensus.cc:504] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:55.713958 5221 tablet_bootstrap.cc:492] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:55.714687 5219 raft_consensus.cc:348] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.715526 5219 raft_consensus.cc:374] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:55.715574 5221 ts_tablet_manager.cc:1397] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.169s user 0.012s sys 0.000s
I20260927 23:28:55.715787 5219 raft_consensus.cc:729] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:55.723618 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 260ab7817da1431ebd80ee7dd00c40b4 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:55.731469 5224 ts_tablet_manager.cc:1918] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:55.713469 5310 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:55.731876 5224 log.cc:1196] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/2dcb78350f4c4b73beada474bcb01169
I20260927 23:28:55.730378 5229 tablet_bootstrap.cc:492] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:55.732354 5310 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:55.732574 5221 raft_consensus.cc:348] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:55.732705 5229 ts_tablet_manager.cc:1397] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.102s user 0.007s sys 0.005s
I20260927 23:28:55.733289 5221 raft_consensus.cc:374] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:55.734019 5221 raft_consensus.cc:729] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:55.734686 5221 consensus_queue.cc:260] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:55.734995 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c7f48abe7db4a5aa39892af9a3e3713" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
W20260927 23:28:55.735291 5051 leader_election.cc:341] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:28:55.735587 4974 raft_consensus.cc:3037] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:55.735899 5310 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:28:55.736456 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:28:55.738488 5229 raft_consensus.cc:348] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:55.739192 5229 raft_consensus.cc:374] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:55.739430 5229 raft_consensus.cc:729] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:55.742395 5296 raft_consensus.cc:482] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:55.742833 5296 raft_consensus.cc:504] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.744663 5296 leader_election.cc:288] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:55.744699 5219 consensus_queue.cc:260] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:55.747399 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:28:55.747874 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:28:55.748656 5310 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:55.749536 4907 leader_election.cc:341] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c
I20260927 23:28:55.751610 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac88a4e4bee64586bef7b6b409a63f4c" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:55.755494 5277 leader_election.cc:288] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:55.755786 5310 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:55.756235 5310 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:55.758821 5310 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:28:55.759541 4910 leader_election.cc:341] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:55.759971 4910 leader_election.cc:302] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
W20260927 23:28:55.761359 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:28:55.761698 5296 raft_consensus.cc:2726] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:55.740049 5229 consensus_queue.cc:260] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:55.765552 4951 tablet_service.cc:1514] Processing DeleteTablet for tablet abb39e8b5d2946ceae480a6a6f9cefb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 7a2477e4ce30497fa94f2dfdb3c31a3d) from {username='slave'} at 127.0.0.1:34620
W20260927 23:28:55.765398 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:28:55.767334 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet abb39e8b5d2946ceae480a6a6f9cefb1 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet abb39e8b5d2946ceae480a6a6f9cefb1 already in progress: creating tablet
I20260927 23:28:55.767606 5054 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:28:55.769253 5277 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:55.755023 4974 raft_consensus.cc:2454] T 0c7f48abe7db4a5aa39892af9a3e3713 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
W20260927 23:28:55.772625 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
I20260927 23:28:55.773332 5054 leader_election.cc:302] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:55.776547 5277 raft_consensus.cc:2781] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:55.789853 5221 ts_tablet_manager.cc:1428] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.072s user 0.005s sys 0.000s
I20260927 23:28:55.790095 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 260ab7817da1431ebd80ee7dd00c40b4
I20260927 23:28:55.792639 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 260ab7817da1431ebd80ee7dd00c40b4 deleted and replaced by tablet c4deca666d4e4d25aae3ed6f13e97c4d
I20260927 23:28:55.793783 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet 260ab7817da1431ebd80ee7dd00c40b4 with delete_type TABLET_DATA_DELETED (replaced by tablet c4deca666d4e4d25aae3ed6f13e97c4d) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:55.794464 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet 260ab7817da1431ebd80ee7dd00c40b4 with delete_type TABLET_DATA_DELETED (replaced by tablet c4deca666d4e4d25aae3ed6f13e97c4d) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:55.796235 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 260ab7817da1431ebd80ee7dd00c40b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 260ab7817da1431ebd80ee7dd00c40b4 already in progress: creating tablet
I20260927 23:28:55.797075 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 2826716e5b184cc2b60d556ca2b89f4d (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.797267 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet 260ab7817da1431ebd80ee7dd00c40b4 with delete_type TABLET_DATA_DELETED (replaced by tablet c4deca666d4e4d25aae3ed6f13e97c4d) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:55.798276 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2826716e5b184cc2b60d556ca2b89f4d. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:55.799005 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 260ab7817da1431ebd80ee7dd00c40b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 260ab7817da1431ebd80ee7dd00c40b4 already in progress: creating tablet
I20260927 23:28:55.803941 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet abb39e8b5d2946ceae480a6a6f9cefb1 with delete_type TABLET_DATA_DELETED (replaced by tablet 7a2477e4ce30497fa94f2dfdb3c31a3d) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:55.804798 5223 tablet_replica.cc:330] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:28:55.805545 5223 raft_consensus.cc:2229] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:55.806017 5223 raft_consensus.cc:2258] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:55.808109 5223 ts_tablet_manager.cc:1905] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:55.808810 5310 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:55.809299 5310 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.810343 5224 ts_tablet_manager.cc:1939] T 2dcb78350f4c4b73beada474bcb01169 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
W20260927 23:28:55.813838 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:28:55.820080 5310 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:55.825227 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 2dcb78350f4c4b73beada474bcb01169 (table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]) successfully deleted
W20260927 23:28:55.822822 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:28:55.826282 5224 tablet_replica.cc:330] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:55.826161 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:28:55.826911 5224 raft_consensus.cc:2229] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:55.827373 5224 raft_consensus.cc:2258] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:55.827533 5310 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:55.809726 5221 tablet_bootstrap.cc:492] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:55.837671 5221 tablet_bootstrap.cc:654] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:55.838141 5277 raft_consensus.cc:686] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:28:55.842183 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 2826716e5b184cc2b60d556ca2b89f4d (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.843431 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2826716e5b184cc2b60d556ca2b89f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.765407 5229 ts_tablet_manager.cc:1428] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.032s user 0.005s sys 0.000s
I20260927 23:28:55.853940 5229 tablet_bootstrap.cc:492] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:55.855650 5224 ts_tablet_manager.cc:1905] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:55.839455 5277 consensus_queue.cc:237] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:55.839195 4952 tablet_service.cc:1467] Processing CreateTablet for tablet 2826716e5b184cc2b60d556ca2b89f4d (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:55.863273 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2826716e5b184cc2b60d556ca2b89f4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:55.836202 5219 ts_tablet_manager.cc:1428] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.153s user 0.006s sys 0.003s
I20260927 23:28:55.887161 5219 tablet_bootstrap.cc:492] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:55.904939 5221 tablet_bootstrap.cc:492] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:55.915248 5229 tablet_bootstrap.cc:654] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:55.916528 5221 ts_tablet_manager.cc:1397] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.107s user 0.010s sys 0.002s
I20260927 23:28:55.946077 5223 ts_tablet_manager.cc:1918] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:55.931062 5221 raft_consensus.cc:348] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:55.954218 5221 raft_consensus.cc:374] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:55.954695 5221 raft_consensus.cc:729] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:55.956023 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 260ab7817da1431ebd80ee7dd00c40b4 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:55.957160 5223 log.cc:1196] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:28:55.960866 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:28:55.963654 5223 ts_tablet_manager.cc:1939] T abb39e8b5d2946ceae480a6a6f9cefb1 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:28:55.956017 5221 consensus_queue.cc:260] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:55.980618 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
W20260927 23:28:55.982995 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
I20260927 23:28:55.986110 5221 ts_tablet_manager.cc:1428] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.066s user 0.006s sys 0.001s
I20260927 23:28:55.989071 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet abb39e8b5d2946ceae480a6a6f9cefb1 (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
I20260927 23:28:55.998185 5221 tablet_bootstrap.cc:492] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:56.007593 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:56.018270 5221 tablet_bootstrap.cc:654] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:56.018570 5224 ts_tablet_manager.cc:1918] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:56.018960 5224 log.cc:1196] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/360070544fca4f62b0aac16d5889e1bc
I20260927 23:28:56.019701 5224 ts_tablet_manager.cc:1939] T 360070544fca4f62b0aac16d5889e1bc P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:56.020941 5224 tablet_replica.cc:330] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:56.021584 5224 raft_consensus.cc:2229] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:56.021363 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 360070544fca4f62b0aac16d5889e1bc (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:28:56.022054 5224 raft_consensus.cc:2258] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:56.027215 5224 ts_tablet_manager.cc:1905] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:56.027302 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 260ab7817da1431ebd80ee7dd00c40b4 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:56.034142 5221 tablet_bootstrap.cc:492] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:56.034615 5221 ts_tablet_manager.cc:1397] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.037s user 0.012s sys 0.005s
I20260927 23:28:56.041352 5221 raft_consensus.cc:348] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.042100 5221 raft_consensus.cc:374] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.042418 5221 raft_consensus.cc:729] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.043635 5224 ts_tablet_manager.cc:1918] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:56.044106 5224 log.cc:1196] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/7a2477e4ce30497fa94f2dfdb3c31a3d
I20260927 23:28:56.045385 5224 ts_tablet_manager.cc:1939] T 7a2477e4ce30497fa94f2dfdb3c31a3d P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:56.045887 5221 consensus_queue.cc:260] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.048262 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 7a2477e4ce30497fa94f2dfdb3c31a3d (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
I20260927 23:28:56.050096 5296 raft_consensus.cc:482] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:56.050575 5296 raft_consensus.cc:504] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.052520 5296 leader_election.cc:288] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:56.054009 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a73bf1fc35439ea3e39ffd843ebfb1" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:56.055394 4910 leader_election.cc:341] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:56.055428 5221 ts_tablet_manager.cc:1428] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.021s user 0.007s sys 0.001s
I20260927 23:28:56.056833 5221 tablet_bootstrap.cc:492] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:56.059381 4907 leader_election.cc:341] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:28:56.059842 4907 leader_election.cc:302] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
W20260927 23:28:56.062985 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
I20260927 23:28:56.064793 5296 raft_consensus.cc:2726] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:56.065220 5229 tablet_bootstrap.cc:492] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:56.065691 5229 ts_tablet_manager.cc:1397] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.212s user 0.008s sys 0.003s
I20260927 23:28:56.072005 4811 tablet_service.cc:1467] Processing CreateTablet for tablet c4deca666d4e4d25aae3ed6f13e97c4d (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.073390 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4deca666d4e4d25aae3ed6f13e97c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.073786 5097 tablet_service.cc:1467] Processing CreateTablet for tablet c4deca666d4e4d25aae3ed6f13e97c4d (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.074957 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4deca666d4e4d25aae3ed6f13e97c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.078769 4952 tablet_service.cc:1467] Processing CreateTablet for tablet c4deca666d4e4d25aae3ed6f13e97c4d (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.080097 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4deca666d4e4d25aae3ed6f13e97c4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.081035 5219 tablet_bootstrap.cc:654] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:56.087980 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet 260ab7817da1431ebd80ee7dd00c40b4 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 260ab7817da1431ebd80ee7dd00c40b4 already in progress: creating tablet
W20260927 23:28:56.088997 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
I20260927 23:28:56.101303 5221 tablet_bootstrap.cc:654] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:56.133392 5229 raft_consensus.cc:348] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:56.134362 5229 raft_consensus.cc:374] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.134688 5229 raft_consensus.cc:729] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.135380 5229 consensus_queue.cc:260] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:56.149315 5277 raft_consensus.cc:482] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:56.149837 5277 raft_consensus.cc:504] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
W20260927 23:28:56.150365 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:28:56.150614 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:56.151877 5277 leader_election.cc:288] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:56.154834 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0c6ad197164aa98d73d357de26b488" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:56.155499 4974 raft_consensus.cc:2454] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
W20260927 23:28:56.156754 5051 leader_election.cc:341] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:56.157567 5054 leader_election.cc:302] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:56.158752 5277 raft_consensus.cc:2781] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:56.159161 5277 raft_consensus.cc:482] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:56.159541 5277 raft_consensus.cc:3037] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:56.166986 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet a007bdab1e3c4e40a259b97f1e43bd51 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:56.168167 5221 tablet_bootstrap.cc:492] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:56.167742 5277 raft_consensus.cc:504] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:56.168743 5221 ts_tablet_manager.cc:1397] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.112s user 0.015s sys 0.020s
I20260927 23:28:56.171813 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a0c6ad197164aa98d73d357de26b488" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:28:56.172456 4974 raft_consensus.cc:3037] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:56.172848 5221 raft_consensus.cc:348] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.173508 5221 raft_consensus.cc:374] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.175196 5221 raft_consensus.cc:729] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.175845 5221 consensus_queue.cc:260] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:56.177104 5051 leader_election.cc:341] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:56.180783 5221 ts_tablet_manager.cc:1428] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260927 23:28:56.181697 5221 tablet_bootstrap.cc:492] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:56.184554 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
I20260927 23:28:56.187124 5221 tablet_bootstrap.cc:654] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:56.191179 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
I20260927 23:28:56.193526 4974 raft_consensus.cc:2454] T 3a0c6ad197164aa98d73d357de26b488 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:28:56.196012 5277 leader_election.cc:288] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:56.200491 5054 leader_election.cc:302] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:56.201910 5277 raft_consensus.cc:2781] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:56.204716 5277 raft_consensus.cc:686] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:28:56.205817 5277 consensus_queue.cc:237] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:56.208967 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:28:56.211244 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet a007bdab1e3c4e40a259b97f1e43bd51 with delete_type TABLET_DATA_DELETED (replaced by tablet 6c3c55f368dd4ba991daded55d2e3069) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:56.211812 4622 catalog_manager.cc:6565] ReplaceTablet: tablet a007bdab1e3c4e40a259b97f1e43bd51 deleted and replaced by tablet 6c3c55f368dd4ba991daded55d2e3069
I20260927 23:28:56.212561 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet a007bdab1e3c4e40a259b97f1e43bd51 with delete_type TABLET_DATA_DELETED (replaced by tablet 6c3c55f368dd4ba991daded55d2e3069) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:56.213199 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet a007bdab1e3c4e40a259b97f1e43bd51 with delete_type TABLET_DATA_DELETED (replaced by tablet 6c3c55f368dd4ba991daded55d2e3069) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:56.214221 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet a007bdab1e3c4e40a259b97f1e43bd51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a007bdab1e3c4e40a259b97f1e43bd51 already in progress: creating tablet
I20260927 23:28:56.218325 5219 tablet_bootstrap.cc:492] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:56.218786 5219 ts_tablet_manager.cc:1397] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.332s user 0.012s sys 0.001s
W20260927 23:28:56.218695 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet a007bdab1e3c4e40a259b97f1e43bd51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a007bdab1e3c4e40a259b97f1e43bd51 already in progress: creating tablet
W20260927 23:28:56.220510 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet a007bdab1e3c4e40a259b97f1e43bd51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a007bdab1e3c4e40a259b97f1e43bd51 already in progress: creating tablet
I20260927 23:28:56.231487 5296 raft_consensus.cc:482] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:56.232133 5296 raft_consensus.cc:504] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.234386 5296 leader_election.cc:288] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:56.234334 5219 raft_consensus.cc:348] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.235031 5219 raft_consensus.cc:374] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.235270 5219 raft_consensus.cc:729] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.241255 5229 ts_tablet_manager.cc:1428] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.175s user 0.005s sys 0.003s
I20260927 23:28:56.242298 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "667f1147ecae4874854935982dd19e96" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:56.242928 4907 leader_election.cc:341] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 667f1147ecae4874854935982dd19e96
I20260927 23:28:56.243381 5221 tablet_bootstrap.cc:492] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
W20260927 23:28:56.244807 4910 leader_election.cc:341] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:56.245211 4910 leader_election.cc:302] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:56.245954 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 2826716e5b184cc2b60d556ca2b89f4d from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:56.246657 5296 raft_consensus.cc:2726] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:56.242287 5229 tablet_bootstrap.cc:492] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:56.254163 5219 consensus_queue.cc:260] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.243872 5221 ts_tablet_manager.cc:1397] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.062s user 0.022s sys 0.033s
W20260927 23:28:56.274080 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:56.273439 5221 raft_consensus.cc:348] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.274386 5221 raft_consensus.cc:374] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.274739 5221 raft_consensus.cc:729] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.275394 5221 consensus_queue.cc:260] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.281088 5221 ts_tablet_manager.cc:1428] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260927 23:28:56.283012 5221 tablet_bootstrap.cc:492] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:56.283092 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:28:56.285871 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
I20260927 23:28:56.287894 5219 ts_tablet_manager.cc:1428] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.069s user 0.007s sys 0.000s
I20260927 23:28:56.288801 5219 tablet_bootstrap.cc:492] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:56.298069 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 2826716e5b184cc2b60d556ca2b89f4d
I20260927 23:28:56.299454 4622 catalog_manager.cc:6565] ReplaceTablet: tablet 2826716e5b184cc2b60d556ca2b89f4d deleted and replaced by tablet 70cb3313148a47f8b6c692222508eb88
I20260927 23:28:56.300570 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet 2826716e5b184cc2b60d556ca2b89f4d with delete_type TABLET_DATA_DELETED (replaced by tablet 70cb3313148a47f8b6c692222508eb88) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:56.302305 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 2826716e5b184cc2b60d556ca2b89f4d with delete_type TABLET_DATA_DELETED (replaced by tablet 70cb3313148a47f8b6c692222508eb88) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:56.302114 5221 tablet_bootstrap.cc:654] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:56.304072 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 2826716e5b184cc2b60d556ca2b89f4d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2826716e5b184cc2b60d556ca2b89f4d already in progress: creating tablet
I20260927 23:28:56.305503 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet 2826716e5b184cc2b60d556ca2b89f4d with delete_type TABLET_DATA_DELETED (replaced by tablet 70cb3313148a47f8b6c692222508eb88) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:56.307093 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet 2826716e5b184cc2b60d556ca2b89f4d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2826716e5b184cc2b60d556ca2b89f4d already in progress: creating tablet
W20260927 23:28:56.307286 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 2826716e5b184cc2b60d556ca2b89f4d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 2826716e5b184cc2b60d556ca2b89f4d already in progress: creating tablet
I20260927 23:28:56.311337 5229 tablet_bootstrap.cc:654] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:56.311643 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
I20260927 23:28:56.315385 4811 tablet_service.cc:1467] Processing CreateTablet for tablet 6c3c55f368dd4ba991daded55d2e3069 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.316624 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c3c55f368dd4ba991daded55d2e3069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.321678 5221 tablet_bootstrap.cc:492] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:56.322134 5221 ts_tablet_manager.cc:1397] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.039s user 0.008s sys 0.008s
I20260927 23:28:56.324550 5221 raft_consensus.cc:348] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.326629 5221 raft_consensus.cc:374] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.326958 5221 raft_consensus.cc:729] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.327864 4952 tablet_service.cc:1467] Processing CreateTablet for tablet 6c3c55f368dd4ba991daded55d2e3069 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.328414 5221 consensus_queue.cc:260] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.329090 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c3c55f368dd4ba991daded55d2e3069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.331735 5221 ts_tablet_manager.cc:1428] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.009s user 0.005s sys 0.003s
I20260927 23:28:56.332741 5221 tablet_bootstrap.cc:492] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:56.338678 5219 tablet_bootstrap.cc:654] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:56.339367 5221 tablet_bootstrap.cc:654] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:56.353332 5221 tablet_bootstrap.cc:492] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:56.353821 5221 ts_tablet_manager.cc:1397] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.008s
I20260927 23:28:56.356158 5221 raft_consensus.cc:348] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.357062 5221 raft_consensus.cc:374] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.357302 5221 raft_consensus.cc:729] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.359092 5229 tablet_bootstrap.cc:492] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:56.357957 5221 consensus_queue.cc:260] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.359292 4951 tablet_service.cc:1514] Processing DeleteTablet for tablet a007bdab1e3c4e40a259b97f1e43bd51 with delete_type TABLET_DATA_DELETED (replaced by tablet 6c3c55f368dd4ba991daded55d2e3069) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:56.359543 5229 ts_tablet_manager.cc:1397] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.117s user 0.006s sys 0.004s
I20260927 23:28:56.360035 5223 tablet_replica.cc:330] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:28:56.360790 5223 raft_consensus.cc:2229] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:56.361214 5223 raft_consensus.cc:2258] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:56.363399 5223 ts_tablet_manager.cc:1905] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:56.369109 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:28:56.372743 5221 ts_tablet_manager.cc:1428] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20260927 23:28:56.374954 5221 tablet_bootstrap.cc:492] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:56.374660 5229 raft_consensus.cc:348] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.375552 5229 raft_consensus.cc:374] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.375857 5229 raft_consensus.cc:729] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.377058 5229 consensus_queue.cc:260] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.382778 5219 tablet_bootstrap.cc:492] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:56.383297 5219 ts_tablet_manager.cc:1397] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.095s user 0.007s sys 0.004s
I20260927 23:28:56.390621 5223 ts_tablet_manager.cc:1918] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260927 23:28:56.391688 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:28:56.392124 5221 tablet_bootstrap.cc:654] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:56.394903 5223 log.cc:1196] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:28:56.395754 5223 ts_tablet_manager.cc:1939] T a007bdab1e3c4e40a259b97f1e43bd51 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:28:56.399142 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet a007bdab1e3c4e40a259b97f1e43bd51 with delete_type TABLET_DATA_DELETED (replaced by tablet 6c3c55f368dd4ba991daded55d2e3069) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:56.399925 5224 tablet_replica.cc:330] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:56.400281 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet a007bdab1e3c4e40a259b97f1e43bd51 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:56.402383 5224 raft_consensus.cc:2229] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:56.402876 5224 raft_consensus.cc:2258] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:56.405463 5224 ts_tablet_manager.cc:1905] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:56.408495 5221 tablet_bootstrap.cc:492] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:56.409037 5221 ts_tablet_manager.cc:1397] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.035s user 0.010s sys 0.004s
I20260927 23:28:56.410631 5219 raft_consensus.cc:348] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.411373 5219 raft_consensus.cc:374] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.411638 5219 raft_consensus.cc:729] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.411540 5221 raft_consensus.cc:348] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.412230 5221 raft_consensus.cc:374] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.412576 5221 raft_consensus.cc:729] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.414196 5221 consensus_queue.cc:260] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.396956 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 6c3c55f368dd4ba991daded55d2e3069 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.422369 5221 ts_tablet_manager.cc:1428] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.013s user 0.006s sys 0.003s
I20260927 23:28:56.422433 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c3c55f368dd4ba991daded55d2e3069. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.422919 5229 ts_tablet_manager.cc:1428] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.063s user 0.007s sys 0.001s
I20260927 23:28:56.423908 5229 tablet_bootstrap.cc:492] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:56.416864 5277 raft_consensus.cc:482] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:56.429693 5229 tablet_bootstrap.cc:654] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:56.429713 5277 raft_consensus.cc:504] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:56.433979 5277 leader_election.cc:288] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:56.412266 5219 consensus_queue.cc:260] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.437624 5219 ts_tablet_manager.cc:1428] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.054s user 0.003s sys 0.002s
I20260927 23:28:56.438588 5219 tablet_bootstrap.cc:492] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:56.439073 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3148da66024c4684d5eaf3d60c8c58" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:56.439829 4974 raft_consensus.cc:2454] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:28:56.441576 5221 tablet_bootstrap.cc:492] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:56.441831 5051 leader_election.cc:341] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:28:56.451992 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:28:56.453199 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 2826716e5b184cc2b60d556ca2b89f4d from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:56.462073 5054 leader_election.cc:302] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:56.464025 5277 raft_consensus.cc:2781] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:56.464475 5277 raft_consensus.cc:482] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:56.464919 5277 raft_consensus.cc:3037] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:56.467330 5221 tablet_bootstrap.cc:654] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:56.481874 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
I20260927 23:28:56.482402 5277 raft_consensus.cc:504] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:56.483860 5219 tablet_bootstrap.cc:654] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:56.486464 5051 leader_election.cc:341] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58
I20260927 23:28:56.487871 5277 leader_election.cc:288] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:56.489257 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a3148da66024c4684d5eaf3d60c8c58" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:28:56.489883 4974 raft_consensus.cc:3037] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:56.494133 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
I20260927 23:28:56.498406 4974 raft_consensus.cc:2454] T 0a3148da66024c4684d5eaf3d60c8c58 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:28:56.499661 5054 leader_election.cc:302] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:56.500849 5310 raft_consensus.cc:482] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:56.501091 5229 tablet_bootstrap.cc:492] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:56.501585 5229 ts_tablet_manager.cc:1397] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.078s user 0.014s sys 0.003s
I20260927 23:28:56.501318 5310 raft_consensus.cc:504] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.503719 5310 leader_election.cc:288] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:56.505647 5229 raft_consensus.cc:348] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.506350 5229 raft_consensus.cc:374] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.506600 5229 raft_consensus.cc:729] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.507242 5229 consensus_queue.cc:260] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.504168 5310 raft_consensus.cc:2781] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
W20260927 23:28:56.514416 5051 leader_election.cc:341] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:28:56.515373 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 260ab7817da1431ebd80ee7dd00c40b4 from {username='slave'} at 127.0.0.1:35136
W20260927 23:28:56.516762 4615 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ac88a4e4bee64586bef7b6b409a63f4c consensus state has no leader
I20260927 23:28:56.520809 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a73bf1fc35439ea3e39ffd843ebfb1" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:56.522298 5219 tablet_bootstrap.cc:492] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:56.522759 5219 ts_tablet_manager.cc:1397] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.084s user 0.010s sys 0.000s
I20260927 23:28:56.525838 4974 raft_consensus.cc:2454] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:28:56.527890 5054 leader_election.cc:302] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:56.530082 5219 raft_consensus.cc:348] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.530821 5219 raft_consensus.cc:374] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.531072 5219 raft_consensus.cc:729] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.531951 5277 raft_consensus.cc:2781] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:56.532331 5277 raft_consensus.cc:482] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:56.533363 5277 raft_consensus.cc:3037] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:56.540891 5219 consensus_queue.cc:260] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.550968 5219 ts_tablet_manager.cc:1428] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.028s user 0.005s sys 0.002s
I20260927 23:28:56.552109 5219 tablet_bootstrap.cc:492] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:56.553699 5310 raft_consensus.cc:686] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:28:56.554940 5310 consensus_queue.cc:237] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:56.569455 5224 ts_tablet_manager.cc:1918] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:56.570135 5224 log.cc:1196] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:28:56.570959 5224 ts_tablet_manager.cc:1939] T a007bdab1e3c4e40a259b97f1e43bd51 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:56.573985 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet a007bdab1e3c4e40a259b97f1e43bd51 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:56.576743 4811 tablet_service.cc:1467] Processing CreateTablet for tablet 70cb3313148a47f8b6c692222508eb88 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.577955 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70cb3313148a47f8b6c692222508eb88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.582437 5296 raft_consensus.cc:482] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:56.582954 5296 raft_consensus.cc:504] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.585043 5296 leader_election.cc:288] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
W20260927 23:28:56.589114 4907 leader_election.cc:341] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:56.592134 5219 tablet_bootstrap.cc:654] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:56.595631 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
I20260927 23:28:56.599118 4952 tablet_service.cc:1467] Processing CreateTablet for tablet 70cb3313148a47f8b6c692222508eb88 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.600399 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70cb3313148a47f8b6c692222508eb88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.558317 5229 ts_tablet_manager.cc:1428] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.056s user 0.007s sys 0.000s
I20260927 23:28:56.577661 5277 raft_consensus.cc:504] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.605247 5229 tablet_bootstrap.cc:492] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:56.605816 5277 leader_election.cc:288] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:56.586846 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc208e8764244ac68b2a73be1b3d957e" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:56.609452 4910 leader_election.cc:341] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:56.609927 4910 leader_election.cc:302] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:56.611357 5296 raft_consensus.cc:2726] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:56.612792 5051 leader_election.cc:341] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:28:56.613922 5229 tablet_bootstrap.cc:654] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:56.618527 5219 tablet_bootstrap.cc:492] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:56.619009 5219 ts_tablet_manager.cc:1397] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.067s user 0.006s sys 0.004s
I20260927 23:28:56.624610 5296 raft_consensus.cc:482] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:56.625159 5296 raft_consensus.cc:504] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.627190 5296 leader_election.cc:288] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:56.630419 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dcb78350f4c4b73beada474bcb01169" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:56.633292 4910 leader_election.cc:341] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:56.634804 4907 leader_election.cc:341] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169
I20260927 23:28:56.635277 4907 leader_election.cc:302] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:56.636631 5296 raft_consensus.cc:2726] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:56.639449 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 111653us
I20260927 23:28:56.641271 5229 tablet_bootstrap.cc:492] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:56.641736 5229 ts_tablet_manager.cc:1397] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.037s user 0.011s sys 0.000s
W20260927 23:28:56.642653 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
I20260927 23:28:56.645973 5219 raft_consensus.cc:348] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.646700 5219 raft_consensus.cc:374] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.646929 5219 raft_consensus.cc:729] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.646894 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7a73bf1fc35439ea3e39ffd843ebfb1" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:28:56.647655 4974 raft_consensus.cc:3037] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:56.647539 5219 consensus_queue.cc:260] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.665330 5219 ts_tablet_manager.cc:1428] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.046s user 0.007s sys 0.000s
I20260927 23:28:56.668098 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet c4deca666d4e4d25aae3ed6f13e97c4d from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:56.680719 5221 tablet_bootstrap.cc:492] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:56.685514 5221 ts_tablet_manager.cc:1397] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.244s user 0.017s sys 0.033s
I20260927 23:28:56.685146 5219 tablet_bootstrap.cc:492] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:56.688066 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 2826716e5b184cc2b60d556ca2b89f4d with delete_type TABLET_DATA_DELETED (replaced by tablet 70cb3313148a47f8b6c692222508eb88) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:56.688928 5224 tablet_replica.cc:330] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:56.689601 5224 raft_consensus.cc:2229] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:56.690023 5224 raft_consensus.cc:2258] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:56.692337 5224 ts_tablet_manager.cc:1905] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:56.695089 5219 tablet_bootstrap.cc:654] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:56.696393 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet c4deca666d4e4d25aae3ed6f13e97c4d
I20260927 23:28:56.644266 5229 raft_consensus.cc:348] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.696941 5229 raft_consensus.cc:374] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.697201 5229 raft_consensus.cc:729] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.698113 4622 catalog_manager.cc:6565] ReplaceTablet: tablet c4deca666d4e4d25aae3ed6f13e97c4d deleted and replaced by tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
I20260927 23:28:56.701928 4951 tablet_service.cc:1514] Processing DeleteTablet for tablet c4deca666d4e4d25aae3ed6f13e97c4d with delete_type TABLET_DATA_DELETED (replaced by tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:56.703442 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet c4deca666d4e4d25aae3ed6f13e97c4d with delete_type TABLET_DATA_DELETED (replaced by tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8) from {username='slave'} at 127.0.0.1:53202
W20260927 23:28:56.704018 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet c4deca666d4e4d25aae3ed6f13e97c4d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c4deca666d4e4d25aae3ed6f13e97c4d already in progress: creating tablet
I20260927 23:28:56.661064 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 70cb3313148a47f8b6c692222508eb88 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.708354 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70cb3313148a47f8b6c692222508eb88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.713620 4974 raft_consensus.cc:2454] T e7a73bf1fc35439ea3e39ffd843ebfb1 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:28:56.714452 5229 consensus_queue.cc:260] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.718212 5095 tablet_service.cc:1514] Processing DeleteTablet for tablet c4deca666d4e4d25aae3ed6f13e97c4d with delete_type TABLET_DATA_DELETED (replaced by tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:56.720326 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet c4deca666d4e4d25aae3ed6f13e97c4d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c4deca666d4e4d25aae3ed6f13e97c4d already in progress: creating tablet
I20260927 23:28:56.660112 5329 raft_consensus.cc:482] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:56.721474 5329 raft_consensus.cc:504] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.729820 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 70cb3313148a47f8b6c692222508eb88 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:56.731324 5329 leader_election.cc:288] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:28:56.733145 5051 leader_election.cc:341] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0
I20260927 23:28:56.734939 5229 ts_tablet_manager.cc:1428] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.093s user 0.004s sys 0.003s
I20260927 23:28:56.741524 5229 tablet_bootstrap.cc:492] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
W20260927 23:28:56.748803 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:28:56.749209 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
I20260927 23:28:56.738149 4973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f8e7644db24be5a758b437a5f471c0" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:56.753510 4973 raft_consensus.cc:2454] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
W20260927 23:28:56.755606 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
I20260927 23:28:56.757798 5054 leader_election.cc:302] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:56.758862 5224 ts_tablet_manager.cc:1918] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:56.759316 5224 log.cc:1196] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/2826716e5b184cc2b60d556ca2b89f4d
I20260927 23:28:56.759996 5224 ts_tablet_manager.cc:1939] T 2826716e5b184cc2b60d556ca2b89f4d P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:56.759852 5329 raft_consensus.cc:2781] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:56.760418 5329 raft_consensus.cc:482] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:56.760794 5329 raft_consensus.cc:3037] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:56.763649 5229 tablet_bootstrap.cc:654] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:56.768683 5296 raft_consensus.cc:482] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:56.769212 5296 raft_consensus.cc:504] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.771345 5296 leader_election.cc:288] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:56.777179 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360070544fca4f62b0aac16d5889e1bc" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:56.779242 4907 leader_election.cc:341] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc
W20260927 23:28:56.780776 4910 leader_election.cc:341] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:56.781216 4910 leader_election.cc:302] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:56.788344 4811 tablet_service.cc:1467] Processing CreateTablet for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.789501 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.790175 5296 raft_consensus.cc:2726] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:56.793331 4952 tablet_service.cc:1467] Processing CreateTablet for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.794534 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.794394 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 2826716e5b184cc2b60d556ca2b89f4d (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
W20260927 23:28:56.802811 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet c4deca666d4e4d25aae3ed6f13e97c4d because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c4deca666d4e4d25aae3ed6f13e97c4d already in progress: creating tablet
I20260927 23:28:56.810542 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 70cb3313148a47f8b6c692222508eb88
I20260927 23:28:56.813431 4622 catalog_manager.cc:6565] ReplaceTablet: tablet 70cb3313148a47f8b6c692222508eb88 deleted and replaced by tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
I20260927 23:28:56.814457 4951 tablet_service.cc:1514] Processing DeleteTablet for tablet 70cb3313148a47f8b6c692222508eb88 with delete_type TABLET_DATA_DELETED (replaced by tablet bbdfdfc23b0c4e3d9224ac2815c6cc65) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:56.814591 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet 70cb3313148a47f8b6c692222508eb88 with delete_type TABLET_DATA_DELETED (replaced by tablet bbdfdfc23b0c4e3d9224ac2815c6cc65) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:56.815882 5054 leader_election.cc:302] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
W20260927 23:28:56.816231 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 70cb3313148a47f8b6c692222508eb88 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 70cb3313148a47f8b6c692222508eb88 already in progress: creating tablet
W20260927 23:28:56.817060 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 57898.6us
I20260927 23:28:56.818876 5310 raft_consensus.cc:2781] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:56.820235 5310 raft_consensus.cc:686] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
W20260927 23:28:56.820727 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet 70cb3313148a47f8b6c692222508eb88 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 70cb3313148a47f8b6c692222508eb88 already in progress: creating tablet
I20260927 23:28:56.821877 5310 consensus_queue.cc:237] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.827800 5221 raft_consensus.cc:348] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.828752 5221 raft_consensus.cc:374] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.829074 5221 raft_consensus.cc:729] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.850863 4810 tablet_service.cc:1467] Processing CreateTablet for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.852169 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.861402 5229 tablet_bootstrap.cc:492] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:56.861896 5229 ts_tablet_manager.cc:1397] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.121s user 0.011s sys 0.006s
I20260927 23:28:56.861815 5095 tablet_service.cc:1514] Processing DeleteTablet for tablet 70cb3313148a47f8b6c692222508eb88 with delete_type TABLET_DATA_DELETED (replaced by tablet bbdfdfc23b0c4e3d9224ac2815c6cc65) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:56.865111 5095 tablet_service.cc:1467] Processing CreateTablet for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.866405 5095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.867960 5229 raft_consensus.cc:348] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:56.869768 5229 raft_consensus.cc:374] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.870682 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.872566 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.870055 5229 raft_consensus.cc:729] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.874123 5229 consensus_queue.cc:260] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
W20260927 23:28:56.881639 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 70cb3313148a47f8b6c692222508eb88 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 70cb3313148a47f8b6c692222508eb88 already in progress: creating tablet
I20260927 23:28:56.883894 5219 tablet_bootstrap.cc:492] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:56.884452 5219 ts_tablet_manager.cc:1397] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.200s user 0.014s sys 0.015s
W20260927 23:28:56.884656 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:28:56.840323 5296 raft_consensus.cc:482] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:56.886418 5296 raft_consensus.cc:504] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.887050 5219 raft_consensus.cc:348] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.887771 5219 raft_consensus.cc:374] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.888033 5229 ts_tablet_manager.cc:1428] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.025s user 0.006s sys 0.000s
I20260927 23:28:56.888024 5219 raft_consensus.cc:729] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.851579 4951 tablet_service.cc:1467] Processing CreateTablet for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:56.889140 4951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:56.893514 5296 leader_election.cc:288] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:56.895934 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38cd408c3dc34fabbab5906286e7580a" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:56.896435 4907 leader_election.cc:341] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a
W20260927 23:28:56.897490 4910 leader_election.cc:341] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:56.897903 4910 leader_election.cc:302] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:56.899124 5296 raft_consensus.cc:2726] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:56.904857 5219 consensus_queue.cc:260] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.913417 5229 tablet_bootstrap.cc:492] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:56.919771 5219 ts_tablet_manager.cc:1428] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.035s user 0.005s sys 0.000s
I20260927 23:28:56.920796 5219 tablet_bootstrap.cc:492] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:56.923992 5229 tablet_bootstrap.cc:654] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:56.935554 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260927 23:28:56.939119 4616 consensus_peers.cc:487] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d -> Peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Couldn't send request to peer d9ed71d97661411891f2d2de4cdc2269. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260927 23:28:56.940119 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:56.941962 5229 tablet_bootstrap.cc:492] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:56.942394 5229 ts_tablet_manager.cc:1397] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.029s user 0.010s sys 0.002s
I20260927 23:28:56.953730 5219 tablet_bootstrap.cc:654] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:56.956830 5221 consensus_queue.cc:260] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.955946 5229 raft_consensus.cc:348] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.958467 5229 raft_consensus.cc:374] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:56.958751 5229 raft_consensus.cc:729] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:56.959324 5229 consensus_queue.cc:260] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.966970 5221 ts_tablet_manager.cc:1428] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.279s user 0.006s sys 0.000s
I20260927 23:28:56.967885 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
I20260927 23:28:56.971905 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 with delete_type TABLET_DATA_DELETED (replaced by tablet 77734e994d5d40dfb3efca344d396956) from {username='slave'} at 127.0.0.1:53202
W20260927 23:28:56.973856 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 already in progress: creating tablet
I20260927 23:28:56.980556 5229 ts_tablet_manager.cc:1428] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.037s user 0.006s sys 0.000s
I20260927 23:28:56.984406 4622 catalog_manager.cc:6565] ReplaceTablet: tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 deleted and replaced by tablet 77734e994d5d40dfb3efca344d396956
W20260927 23:28:56.987326 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
I20260927 23:28:56.993125 5310 consensus_queue.cc:1035] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:56.995049 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 with delete_type TABLET_DATA_DELETED (replaced by tablet 77734e994d5d40dfb3efca344d396956) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:56.996264 5229 tablet_bootstrap.cc:492] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:56.996357 5329 raft_consensus.cc:504] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:56.998260 5329 leader_election.cc:288] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:28:56.999770 5051 leader_election.cc:341] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0
I20260927 23:28:57.000671 5219 tablet_bootstrap.cc:492] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:57.001160 5219 ts_tablet_manager.cc:1397] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.081s user 0.010s sys 0.000s
I20260927 23:28:57.003708 5219 raft_consensus.cc:348] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.004510 5229 tablet_bootstrap.cc:654] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:57.004647 5219 raft_consensus.cc:374] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:57.004908 5219 raft_consensus.cc:729] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.005663 5219 consensus_queue.cc:260] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.017459 5219 ts_tablet_manager.cc:1428] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260927 23:28:57.024812 5219 tablet_bootstrap.cc:492] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:57.030614 5219 tablet_bootstrap.cc:654] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:56.984587 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 with delete_type TABLET_DATA_DELETED (replaced by tablet 77734e994d5d40dfb3efca344d396956) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:57.015655 4973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43f8e7644db24be5a758b437a5f471c0" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:28:57.034042 4973 raft_consensus.cc:3037] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:57.042268 4973 raft_consensus.cc:2454] T 43f8e7644db24be5a758b437a5f471c0 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:28:57.044344 5221 tablet_bootstrap.cc:492] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:57.012473 4617 catalog_manager.cc:4808] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 because the tablet was not found. No further retry: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65
W20260927 23:28:57.048064 4617 catalog_manager.cc:4808] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 because the tablet was not found. No further retry: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65
I20260927 23:28:57.058745 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet c4deca666d4e4d25aae3ed6f13e97c4d with delete_type TABLET_DATA_DELETED (replaced by tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:57.059551 5224 tablet_replica.cc:330] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:57.060247 5224 raft_consensus.cc:2229] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:57.060727 5224 raft_consensus.cc:2258] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:57.060856 5229 tablet_bootstrap.cc:492] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:57.061293 5229 ts_tablet_manager.cc:1397] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.065s user 0.005s sys 0.012s
I20260927 23:28:57.062542 5224 ts_tablet_manager.cc:1905] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:57.064769 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:28:57.065162 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
I20260927 23:28:57.066507 5229 raft_consensus.cc:348] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.067193 5229 raft_consensus.cc:374] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260927 23:28:57.068153 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:57.069007 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:57.067444 5229 raft_consensus.cc:729] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.070705 5229 consensus_queue.cc:260] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.076174 5229 ts_tablet_manager.cc:1428] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.015s user 0.004s sys 0.002s
I20260927 23:28:57.079684 5229 tablet_bootstrap.cc:492] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:57.080397 5054 leader_election.cc:302] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:57.081316 5310 raft_consensus.cc:2781] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:57.081758 5310 raft_consensus.cc:686] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:28:57.082587 5310 consensus_queue.cc:237] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.084090 5296 raft_consensus.cc:482] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:57.084623 5296 raft_consensus.cc:504] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.086840 5296 leader_election.cc:288] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:57.089576 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a483b5e32d55406bb3342af93f755028" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:57.090973 4910 leader_election.cc:341] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:57.092096 4907 leader_election.cc:341] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a483b5e32d55406bb3342af93f755028
I20260927 23:28:57.092543 4907 leader_election.cc:302] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:57.094743 5296 raft_consensus.cc:2726] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:57.097505 5229 tablet_bootstrap.cc:654] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:57.105362 5224 ts_tablet_manager.cc:1918] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:57.105829 5224 log.cc:1196] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/c4deca666d4e4d25aae3ed6f13e97c4d
W20260927 23:28:57.108028 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
W20260927 23:28:57.114568 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:28:57.115852 5231 raft_consensus.cc:482] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:57.116552 5231 raft_consensus.cc:504] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.117564 5219 tablet_bootstrap.cc:492] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:57.118743 5231 leader_election.cc:288] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:57.120065 5224 ts_tablet_manager.cc:1939] T c4deca666d4e4d25aae3ed6f13e97c4d P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:57.118570 5219 ts_tablet_manager.cc:1397] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.100s user 0.006s sys 0.005s
I20260927 23:28:57.120985 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260ab7817da1431ebd80ee7dd00c40b4" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:57.121389 4973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260ab7817da1431ebd80ee7dd00c40b4" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:57.122764 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet c4deca666d4e4d25aae3ed6f13e97c4d (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
W20260927 23:28:57.122915 4769 leader_election.cc:341] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:57.123574 4769 leader_election.cc:341] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:57.123925 4769 leader_election.cc:302] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:57.124651 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:57.125025 5231 raft_consensus.cc:2726] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:57.138871 5219 raft_consensus.cc:348] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.153512 5219 raft_consensus.cc:374] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:57.153862 5219 raft_consensus.cc:729] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.154661 5219 consensus_queue.cc:260] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.157410 5219 ts_tablet_manager.cc:1428] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.036s user 0.005s sys 0.000s
I20260927 23:28:57.158406 5219 tablet_bootstrap.cc:492] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:57.164119 5219 tablet_bootstrap.cc:654] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:57.179561 5221 tablet_bootstrap.cc:654] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:57.172066 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:28:57.211972 5231 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:57.212564 5231 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.213085 4811 tablet_service.cc:1467] Processing CreateTablet for tablet 77734e994d5d40dfb3efca344d396956 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:57.214394 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77734e994d5d40dfb3efca344d396956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:57.216769 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5552715138046279f00dcf497d05a72" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:57.217269 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c5552715138046279f00dcf497d05a72" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:57.214632 5231 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:57.218196 4951 tablet_service.cc:1467] Processing CreateTablet for tablet 77734e994d5d40dfb3efca344d396956 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:57.215512 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet 70cb3313148a47f8b6c692222508eb88 with delete_type TABLET_DATA_DELETED (replaced by tablet bbdfdfc23b0c4e3d9224ac2815c6cc65) from {username='slave'} at 127.0.0.1:53202
W20260927 23:28:57.218624 4769 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:57.219859 5224 tablet_replica.cc:330] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
W20260927 23:28:57.220091 4769 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:57.220603 5224 raft_consensus.cc:2229] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:57.221051 5224 raft_consensus.cc:2258] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:57.224135 5224 ts_tablet_manager.cc:1905] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:57.227209 5219 tablet_bootstrap.cc:492] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:57.220618 4769 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:57.227690 5219 ts_tablet_manager.cc:1397] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.069s user 0.013s sys 0.003s
I20260927 23:28:57.228410 5231 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:57.229138 5095 tablet_service.cc:1467] Processing CreateTablet for tablet 77734e994d5d40dfb3efca344d396956 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:57.230436 5095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77734e994d5d40dfb3efca344d396956. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:57.224236 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
W20260927 23:28:57.233106 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:57.233393 4951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77734e994d5d40dfb3efca344d396956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:57.238704 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 6c3c55f368dd4ba991daded55d2e3069 from {username='slave'} at 127.0.0.1:35136
W20260927 23:28:57.247151 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:28:57.249830 5221 tablet_bootstrap.cc:492] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:57.250367 5221 ts_tablet_manager.cc:1397] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.206s user 0.006s sys 0.003s
I20260927 23:28:57.254351 5219 raft_consensus.cc:348] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.255110 5219 raft_consensus.cc:374] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:57.255358 5219 raft_consensus.cc:729] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.258884 5229 tablet_bootstrap.cc:492] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:57.259416 5229 ts_tablet_manager.cc:1397] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.180s user 0.025s sys 0.016s
I20260927 23:28:57.261220 5224 ts_tablet_manager.cc:1918] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:57.262872 5229 raft_consensus.cc:348] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.263652 5229 raft_consensus.cc:374] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:57.263917 5224 log.cc:1196] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/70cb3313148a47f8b6c692222508eb88
I20260927 23:28:57.263916 5229 raft_consensus.cc:729] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.264911 5224 ts_tablet_manager.cc:1939] T 70cb3313148a47f8b6c692222508eb88 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:57.275852 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 70cb3313148a47f8b6c692222508eb88 (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
I20260927 23:28:57.275822 5221 raft_consensus.cc:348] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.276615 5221 raft_consensus.cc:374] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:57.276889 5221 raft_consensus.cc:729] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.274257 5229 consensus_queue.cc:260] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.277704 5219 consensus_queue.cc:260] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.282739 5229 ts_tablet_manager.cc:1428] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.023s user 0.000s sys 0.006s
I20260927 23:28:57.285431 5219 ts_tablet_manager.cc:1428] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.057s user 0.008s sys 0.000s
W20260927 23:28:57.288496 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
I20260927 23:28:57.304637 5229 tablet_bootstrap.cc:492] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:57.300664 5221 consensus_queue.cc:260] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.286497 5219 tablet_bootstrap.cc:492] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:57.318831 5221 ts_tablet_manager.cc:1428] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.068s user 0.008s sys 0.000s
I20260927 23:28:57.322470 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 6c3c55f368dd4ba991daded55d2e3069
I20260927 23:28:57.324661 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c3c55f368dd4ba991daded55d2e3069 with delete_type TABLET_DATA_DELETED (replaced by tablet 64cfbad1561a41048009b5fc819cc5de) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:57.325433 5224 tablet_replica.cc:330] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:57.325731 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 6c3c55f368dd4ba991daded55d2e3069 deleted and replaced by tablet 64cfbad1561a41048009b5fc819cc5de
I20260927 23:28:57.326215 5224 raft_consensus.cc:2229] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:57.326668 5224 raft_consensus.cc:2258] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:57.329067 5224 ts_tablet_manager.cc:1905] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:57.325258 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c3c55f368dd4ba991daded55d2e3069 with delete_type TABLET_DATA_DELETED (replaced by tablet 64cfbad1561a41048009b5fc819cc5de) from {username='slave'} at 127.0.0.1:34620
W20260927 23:28:57.332965 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 6c3c55f368dd4ba991daded55d2e3069 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6c3c55f368dd4ba991daded55d2e3069 already in progress: creating tablet
I20260927 23:28:57.342530 5221 tablet_bootstrap.cc:492] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:57.345351 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c3c55f368dd4ba991daded55d2e3069 with delete_type TABLET_DATA_DELETED (replaced by tablet 64cfbad1561a41048009b5fc819cc5de) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:57.346879 5329 raft_consensus.cc:482] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:28:57.347555 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 6c3c55f368dd4ba991daded55d2e3069 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6c3c55f368dd4ba991daded55d2e3069 already in progress: creating tablet
I20260927 23:28:57.347350 5329 raft_consensus.cc:504] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.350128 5329 leader_election.cc:288] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:57.351135 5229 tablet_bootstrap.cc:654] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:57.352370 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc208e8764244ac68b2a73be1b3d957e" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:57.353092 4974 raft_consensus.cc:2454] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
W20260927 23:28:57.354064 5051 leader_election.cc:341] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:57.355329 5054 leader_election.cc:302] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:57.356575 5329 raft_consensus.cc:2781] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:57.356899 5329 raft_consensus.cc:482] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:57.357178 5329 raft_consensus.cc:3037] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:57.365293 5219 tablet_bootstrap.cc:654] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:57.328424 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
I20260927 23:28:57.369401 5221 tablet_bootstrap.cc:654] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:57.372081 5224 ts_tablet_manager.cc:1918] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:57.373680 5224 log.cc:1196] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/6c3c55f368dd4ba991daded55d2e3069
I20260927 23:28:57.375432 5224 ts_tablet_manager.cc:1939] T 6c3c55f368dd4ba991daded55d2e3069 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:57.378075 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 6c3c55f368dd4ba991daded55d2e3069 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:28:57.379704 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 with delete_type TABLET_DATA_DELETED (replaced by tablet 77734e994d5d40dfb3efca344d396956) from {username='slave'} at 127.0.0.1:53202
W20260927 23:28:57.383708 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
I20260927 23:28:57.386327 5224 tablet_replica.cc:330] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:57.391782 5310 raft_consensus.cc:482] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:57.392369 5310 raft_consensus.cc:504] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.397006 5224 raft_consensus.cc:2229] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:57.397738 5224 raft_consensus.cc:2258] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:57.398698 5310 leader_election.cc:288] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:57.399617 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38cd408c3dc34fabbab5906286e7580a" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:28:57.400738 5051 leader_election.cc:341] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a
I20260927 23:28:57.400317 4974 raft_consensus.cc:2454] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:28:57.403205 5054 leader_election.cc:302] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:57.404228 5310 raft_consensus.cc:2781] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:57.404505 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 77734e994d5d40dfb3efca344d396956 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:57.403041 5329 raft_consensus.cc:504] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.404646 5310 raft_consensus.cc:482] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:57.407091 5224 ts_tablet_manager.cc:1905] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:57.407094 5310 raft_consensus.cc:3037] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:57.408854 5051 leader_election.cc:341] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:57.414359 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc208e8764244ac68b2a73be1b3d957e" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:28:57.415069 4974 raft_consensus.cc:3037] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:57.421887 5329 leader_election.cc:288] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:57.425787 4974 raft_consensus.cc:2454] T cc208e8764244ac68b2a73be1b3d957e P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:28:57.427301 5054 leader_election.cc:302] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:57.428961 5329 raft_consensus.cc:2781] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:57.429471 5329 raft_consensus.cc:686] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:28:57.433604 5219 tablet_bootstrap.cc:492] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:57.434074 5219 ts_tablet_manager.cc:1397] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.148s user 0.011s sys 0.000s
I20260927 23:28:57.431746 5329 consensus_queue.cc:237] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.451917 5277 raft_consensus.cc:482] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:57.452603 5277 raft_consensus.cc:504] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.454702 5277 leader_election.cc:288] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:57.455587 5221 tablet_bootstrap.cc:492] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:57.456045 5221 ts_tablet_manager.cc:1397] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.117s user 0.011s sys 0.001s
I20260927 23:28:57.456960 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac88a4e4bee64586bef7b6b409a63f4c" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:28:57.457216 5051 leader_election.cc:341] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c
I20260927 23:28:57.457703 4974 raft_consensus.cc:2454] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:28:57.458948 5054 leader_election.cc:302] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:57.459086 5221 raft_consensus.cc:348] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.459793 5221 raft_consensus.cc:374] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:57.460050 5221 raft_consensus.cc:729] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.460471 5277 raft_consensus.cc:2781] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:57.460812 5277 raft_consensus.cc:482] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:57.461102 5277 raft_consensus.cc:3037] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:57.465710 5229 tablet_bootstrap.cc:492] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:57.466233 5229 ts_tablet_manager.cc:1397] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.162s user 0.008s sys 0.003s
I20260927 23:28:57.469332 5229 raft_consensus.cc:348] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.470108 5229 raft_consensus.cc:374] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:57.470443 5229 raft_consensus.cc:729] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.470260 5277 raft_consensus.cc:504] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.471140 5229 consensus_queue.cc:260] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.472229 5224 ts_tablet_manager.cc:1918] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:57.472703 5224 log.cc:1196] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/bbdfdfc23b0c4e3d9224ac2815c6cc65
I20260927 23:28:57.473371 5224 ts_tablet_manager.cc:1939] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:57.475360 5310 raft_consensus.cc:504] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.478488 5221 consensus_queue.cc:260] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.498379 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet bbdfdfc23b0c4e3d9224ac2815c6cc65 (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
I20260927 23:28:57.499666 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 77734e994d5d40dfb3efca344d396956
I20260927 23:28:57.501114 5296 raft_consensus.cc:482] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:57.501616 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 77734e994d5d40dfb3efca344d396956 with delete_type TABLET_DATA_DELETED (replaced by tablet 6846550519ce44fdafbc895ddca98d8b) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:57.501624 5296 raft_consensus.cc:504] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.503156 4622 catalog_manager.cc:6565] ReplaceTablet: tablet 77734e994d5d40dfb3efca344d396956 deleted and replaced by tablet 6846550519ce44fdafbc895ddca98d8b
I20260927 23:28:57.503562 5296 leader_election.cc:288] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:57.505491 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet 77734e994d5d40dfb3efca344d396956 with delete_type TABLET_DATA_DELETED (replaced by tablet 6846550519ce44fdafbc895ddca98d8b) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:57.509137 4811 tablet_service.cc:1467] Processing CreateTablet for tablet 64cfbad1561a41048009b5fc819cc5de (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:57.510344 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64cfbad1561a41048009b5fc819cc5de. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:57.512777 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet 77734e994d5d40dfb3efca344d396956 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 77734e994d5d40dfb3efca344d396956 already in progress: creating tablet
I20260927 23:28:57.515825 4952 tablet_service.cc:1467] Processing CreateTablet for tablet 64cfbad1561a41048009b5fc819cc5de (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:57.517019 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64cfbad1561a41048009b5fc819cc5de. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:57.523192 4617 catalog_manager.cc:4808] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 77734e994d5d40dfb3efca344d396956 because the tablet was not found. No further retry: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956
I20260927 23:28:57.523566 5221 ts_tablet_manager.cc:1428] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.067s user 0.009s sys 0.000s
W20260927 23:28:57.524636 4907 leader_election.cc:341] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d
I20260927 23:28:57.541451 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:57.543195 5221 tablet_bootstrap.cc:492] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:57.563809 5221 tablet_bootstrap.cc:654] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:57.481002 5353 consensus_queue.cc:1035] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:57.506323 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2477e4ce30497fa94f2dfdb3c31a3d" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:57.527380 5229 ts_tablet_manager.cc:1428] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.061s user 0.004s sys 0.003s
I20260927 23:28:57.568694 5229 tablet_bootstrap.cc:492] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:57.570079 5310 leader_election.cc:288] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:28:57.570380 4910 leader_election.cc:341] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:57.570848 4910 leader_election.cc:302] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:57.571600 5219 raft_consensus.cc:348] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.573050 5296 raft_consensus.cc:2726] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:57.583635 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
I20260927 23:28:57.584959 4622 catalog_manager.cc:6565] ReplaceTablet: tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 deleted and replaced by tablet bdecd389ff0341f4ab867eb160786ea9
I20260927 23:28:57.586902 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 with delete_type TABLET_DATA_DELETED (replaced by tablet bdecd389ff0341f4ab867eb160786ea9) from {username='slave'} at 127.0.0.1:53202
W20260927 23:28:57.589318 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 already in progress: creating tablet
I20260927 23:28:57.590504 5219 raft_consensus.cc:374] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:57.590924 5219 raft_consensus.cc:729] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.591689 5219 consensus_queue.cc:260] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.599449 4954 tablet_service.cc:1467] Processing CreateTablet for tablet 6846550519ce44fdafbc895ddca98d8b (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:57.600723 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6846550519ce44fdafbc895ddca98d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:57.614650 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 6846550519ce44fdafbc895ddca98d8b (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:57.615939 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6846550519ce44fdafbc895ddca98d8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:57.619218 5221 tablet_bootstrap.cc:492] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:57.619688 5221 ts_tablet_manager.cc:1397] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.077s user 0.004s sys 0.007s
I20260927 23:28:57.626088 5221 raft_consensus.cc:348] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.626833 5221 raft_consensus.cc:374] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:57.627130 5221 raft_consensus.cc:729] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.627926 5221 consensus_queue.cc:260] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.629670 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 with delete_type TABLET_DATA_DELETED (replaced by tablet bdecd389ff0341f4ab867eb160786ea9) from {username='slave'} at 127.0.0.1:53202
W20260927 23:28:57.631464 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 already in progress: creating tablet
I20260927 23:28:57.639880 4953 tablet_service.cc:1467] Processing CreateTablet for tablet bdecd389ff0341f4ab867eb160786ea9 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:57.641110 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdecd389ff0341f4ab867eb160786ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:57.642758 5221 ts_tablet_manager.cc:1428] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260927 23:28:57.646283 5221 tablet_bootstrap.cc:492] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:57.646196 4812 tablet_service.cc:1467] Processing CreateTablet for tablet bdecd389ff0341f4ab867eb160786ea9 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:57.647392 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdecd389ff0341f4ab867eb160786ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:57.665271 5221 tablet_bootstrap.cc:654] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:57.565001 5051 leader_election.cc:341] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c
W20260927 23:28:57.545467 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 75346.7us
W20260927 23:28:57.684815 5051 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:40571, user_credentials={real_user=slave}} blocked reactor thread for 119867us
W20260927 23:28:57.687577 5051 leader_election.cc:341] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a
I20260927 23:28:57.576711 5229 tablet_bootstrap.cc:654] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:57.689944 5277 leader_election.cc:288] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:57.690783 5277 raft_consensus.cc:482] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:57.691273 5277 raft_consensus.cc:504] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.692559 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 with delete_type TABLET_DATA_DELETED (replaced by tablet bdecd389ff0341f4ab867eb160786ea9) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:57.693773 5096 tablet_service.cc:1467] Processing CreateTablet for tablet 6846550519ce44fdafbc895ddca98d8b (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:57.694850 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6846550519ce44fdafbc895ddca98d8b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:57.697824 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:57.704815 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 64cfbad1561a41048009b5fc819cc5de (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:57.706182 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 64cfbad1561a41048009b5fc819cc5de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:57.710285 5219 ts_tablet_manager.cc:1428] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.276s user 0.006s sys 0.000s
I20260927 23:28:57.722321 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:57.721849 5277 leader_election.cc:288] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:28:57.723182 5051 leader_election.cc:341] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 667f1147ecae4874854935982dd19e96
I20260927 23:28:57.722069 5094 tablet_service.cc:1467] Processing CreateTablet for tablet bdecd389ff0341f4ab867eb160786ea9 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:57.728951 5219 tablet_bootstrap.cc:492] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:57.730756 5094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bdecd389ff0341f4ab867eb160786ea9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:57.734542 5219 tablet_bootstrap.cc:654] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:57.736922 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 already in progress: creating tablet
I20260927 23:28:57.738945 5310 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:57.739403 5310 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:57.742779 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:57.743986 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
I20260927 23:28:57.744494 5310 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:28:57.747128 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:28:57.756707 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 with delete_type TABLET_DATA_DELETED (replaced by tablet bdecd389ff0341f4ab867eb160786ea9) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:57.757526 5224 tablet_replica.cc:330] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:57.758185 5224 raft_consensus.cc:2229] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:57.758632 5224 raft_consensus.cc:2258] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:57.765877 5095 tablet_service.cc:1514] Processing DeleteTablet for tablet 77734e994d5d40dfb3efca344d396956 with delete_type TABLET_DATA_DELETED (replaced by tablet 6846550519ce44fdafbc895ddca98d8b) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:57.768036 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 77734e994d5d40dfb3efca344d396956 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 77734e994d5d40dfb3efca344d396956 already in progress: creating tablet
I20260927 23:28:57.768953 5224 ts_tablet_manager.cc:1905] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:57.795804 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet bdecd389ff0341f4ab867eb160786ea9 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:57.813439 4951 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 with delete_type TABLET_DATA_DELETED (replaced by tablet bdecd389ff0341f4ab867eb160786ea9) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:57.819446 5229 tablet_bootstrap.cc:492] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:57.819895 5229 ts_tablet_manager.cc:1397] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.251s user 0.014s sys 0.001s
I20260927 23:28:57.820333 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "667f1147ecae4874854935982dd19e96" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:57.821681 4970 raft_consensus.cc:2454] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:28:57.822530 5229 raft_consensus.cc:348] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.823163 5229 raft_consensus.cc:374] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:57.823385 5229 raft_consensus.cc:729] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.823137 4971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac88a4e4bee64586bef7b6b409a63f4c" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:28:57.823851 4971 raft_consensus.cc:3037] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:57.828764 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:28:57.829234 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:28:57.830222 5310 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:57.831660 5054 leader_election.cc:302] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:57.823967 5229 consensus_queue.cc:260] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.832976 5329 raft_consensus.cc:2781] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:57.833282 5329 raft_consensus.cc:482] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:57.833570 5329 raft_consensus.cc:3037] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:57.836131 5353 raft_consensus.cc:482] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:57.836556 5353 raft_consensus.cc:504] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.838471 5353 leader_election.cc:288] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:57.840490 5329 raft_consensus.cc:504] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.842199 5329 leader_election.cc:288] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:57.842669 5277 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:57.843106 5277 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:57.844877 5277 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:28:57.857554 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:28:57.869228 4969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "667f1147ecae4874854935982dd19e96" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:28:57.869958 4969 raft_consensus.cc:3037] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:57.874495 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dcb78350f4c4b73beada474bcb01169" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:57.875473 4970 raft_consensus.cc:2454] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
W20260927 23:28:57.856634 5051 leader_election.cc:341] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169
W20260927 23:28:57.883260 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:28:57.883885 5051 leader_election.cc:341] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 667f1147ecae4874854935982dd19e96
I20260927 23:28:57.880810 5229 ts_tablet_manager.cc:1428] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.061s user 0.006s sys 0.000s
W20260927 23:28:57.885620 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:57.887554 5219 tablet_bootstrap.cc:492] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:57.888022 5219 ts_tablet_manager.cc:1397] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.159s user 0.014s sys 0.059s
W20260927 23:28:57.889892 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
W20260927 23:28:57.892978 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:28:57.893556 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:28:57.894379 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet bdecd389ff0341f4ab867eb160786ea9
I20260927 23:28:57.894788 5277 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:57.896600 4622 catalog_manager.cc:6565] ReplaceTablet: tablet bdecd389ff0341f4ab867eb160786ea9 deleted and replaced by tablet 0d90eb9269884933a9af575b5af8a47e
I20260927 23:28:57.897909 5095 tablet_service.cc:1514] Processing DeleteTablet for tablet bdecd389ff0341f4ab867eb160786ea9 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d90eb9269884933a9af575b5af8a47e) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:57.899596 4617 catalog_manager.cc:4808] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet bdecd389ff0341f4ab867eb160786ea9 because the tablet was not found. No further retry: Not found: Tablet not found: bdecd389ff0341f4ab867eb160786ea9
I20260927 23:28:57.900391 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet bdecd389ff0341f4ab867eb160786ea9 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d90eb9269884933a9af575b5af8a47e) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:57.903642 5229 tablet_bootstrap.cc:492] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:57.908886 5054 leader_election.cc:302] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:57.910147 5277 raft_consensus.cc:2781] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:57.910522 5277 raft_consensus.cc:482] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:57.910807 5277 raft_consensus.cc:3037] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:57.911530 5229 tablet_bootstrap.cc:654] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:57.921270 5224 ts_tablet_manager.cc:1918] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:57.921667 5224 log.cc:1196] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/0c8328db65eb43ebb79cd9d7a1ddb0d8
I20260927 23:28:57.922245 5224 ts_tablet_manager.cc:1939] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:57.926180 5219 raft_consensus.cc:348] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.917541 5277 raft_consensus.cc:504] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.928754 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "38cd408c3dc34fabbab5906286e7580a" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:28:57.929451 4974 raft_consensus.cc:3037] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:57.929651 5277 leader_election.cc:288] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:28:57.931161 5051 leader_election.cc:341] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169
I20260927 23:28:57.939210 5219 raft_consensus.cc:374] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:57.939893 5219 raft_consensus.cc:729] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.943243 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet c5552715138046279f00dcf497d05a72 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:57.943499 5221 tablet_bootstrap.cc:492] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:57.944116 5221 ts_tablet_manager.cc:1397] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.298s user 0.011s sys 0.001s
I20260927 23:28:57.944664 5219 consensus_queue.cc:260] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.946764 5277 consensus_queue.cc:1035] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:57.947839 5219 ts_tablet_manager.cc:1428] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.059s user 0.006s sys 0.000s
I20260927 23:28:57.951565 5329 consensus_queue.cc:1035] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:57.952847 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet bdecd389ff0341f4ab867eb160786ea9 with delete_type TABLET_DATA_DELETED (replaced by tablet 0d90eb9269884933a9af575b5af8a47e) from {username='slave'} at 127.0.0.1:34620
W20260927 23:28:57.954958 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 already in progress: creating tablet
I20260927 23:28:57.955209 5221 raft_consensus.cc:348] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.956094 5221 raft_consensus.cc:374] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:57.956387 5221 raft_consensus.cc:729] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:57.957067 5221 consensus_queue.cc:260] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.959435 5219 tablet_bootstrap.cc:492] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:57.967185 5329 consensus_queue.cc:1035] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:28:57.969790 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet bdecd389ff0341f4ab867eb160786ea9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bdecd389ff0341f4ab867eb160786ea9 already in progress: creating tablet
W20260927 23:28:57.975680 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
W20260927 23:28:57.977573 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
I20260927 23:28:57.981055 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:28:57.985247 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 77734e994d5d40dfb3efca344d396956 with delete_type TABLET_DATA_DELETED (replaced by tablet 6846550519ce44fdafbc895ddca98d8b) from {username='slave'} at 127.0.0.1:53202
W20260927 23:28:57.986897 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet bdecd389ff0341f4ab867eb160786ea9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet bdecd389ff0341f4ab867eb160786ea9 already in progress: creating tablet
I20260927 23:28:57.989502 5229 tablet_bootstrap.cc:492] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:57.990156 5224 tablet_replica.cc:330] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:57.995198 5224 raft_consensus.cc:2229] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:57.995774 5229 ts_tablet_manager.cc:1397] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.092s user 0.010s sys 0.005s
I20260927 23:28:57.996235 5224 raft_consensus.cc:2258] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:57.998659 5229 raft_consensus.cc:348] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:57.971772 5219 tablet_bootstrap.cc:654] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:58.001896 5224 ts_tablet_manager.cc:1905] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:58.015000 4969 raft_consensus.cc:2454] T 667f1147ecae4874854935982dd19e96 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:28:58.017079 5229 raft_consensus.cc:374] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:58.017540 5229 raft_consensus.cc:729] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:58.018198 5054 leader_election.cc:302] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:58.019701 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dcb78350f4c4b73beada474bcb01169" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:28:58.020504 4970 raft_consensus.cc:3037] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:58.019209 5329 raft_consensus.cc:2781] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:58.021956 5329 raft_consensus.cc:686] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:28:58.018738 5229 consensus_queue.cc:260] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.022744 5329 consensus_queue.cc:237] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.000028 4971 raft_consensus.cc:2454] T ac88a4e4bee64586bef7b6b409a63f4c P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:28:58.032732 5277 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:58.033224 5277 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.034291 5054 leader_election.cc:302] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:58.035159 5277 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:58.035607 5277 raft_consensus.cc:2781] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:58.031610 5229 ts_tablet_manager.cc:1428] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.035s user 0.005s sys 0.001s
I20260927 23:28:58.036278 5277 raft_consensus.cc:686] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:28:58.037109 5277 consensus_queue.cc:237] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.038951 4974 raft_consensus.cc:2454] T 38cd408c3dc34fabbab5906286e7580a P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:28:58.039354 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet c5552715138046279f00dcf497d05a72
W20260927 23:28:58.040661 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:28:58.041371 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet c5552715138046279f00dcf497d05a72 with delete_type TABLET_DATA_DELETED (replaced by tablet 9dffe7e40ecb40c68a06f0cf43861777) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:58.042709 4627 catalog_manager.cc:6565] ReplaceTablet: tablet c5552715138046279f00dcf497d05a72 deleted and replaced by tablet 9dffe7e40ecb40c68a06f0cf43861777
I20260927 23:28:58.045446 5096 tablet_service.cc:1514] Processing DeleteTablet for tablet c5552715138046279f00dcf497d05a72 with delete_type TABLET_DATA_DELETED (replaced by tablet 9dffe7e40ecb40c68a06f0cf43861777) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:58.046335 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:28:58.046710 5051 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
W20260927 23:28:58.047222 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet c5552715138046279f00dcf497d05a72 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c5552715138046279f00dcf497d05a72 already in progress: creating tablet
I20260927 23:28:58.044322 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet c5552715138046279f00dcf497d05a72 with delete_type TABLET_DATA_DELETED (replaced by tablet 9dffe7e40ecb40c68a06f0cf43861777) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:58.052505 5353 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:58.052665 5219 tablet_bootstrap.cc:492] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:58.053017 5223 tablet_replica.cc:330] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:28:58.053045 5219 ts_tablet_manager.cc:1397] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.094s user 0.009s sys 0.001s
I20260927 23:28:58.053190 4812 tablet_service.cc:1467] Processing CreateTablet for tablet 0d90eb9269884933a9af575b5af8a47e (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.053810 5223 raft_consensus.cc:2229] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:58.054206 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d90eb9269884933a9af575b5af8a47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:58.054260 5223 raft_consensus.cc:2258] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:58.055485 5219 raft_consensus.cc:348] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.056147 5219 raft_consensus.cc:374] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:58.056402 5219 raft_consensus.cc:729] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:58.056139 5223 ts_tablet_manager.cc:1905] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:58.063796 5229 tablet_bootstrap.cc:492] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:58.065124 5054 leader_election.cc:302] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:58.066048 5277 raft_consensus.cc:2781] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:58.068596 5277 raft_consensus.cc:686] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:28:58.068522 4970 raft_consensus.cc:2454] T 2dcb78350f4c4b73beada474bcb01169 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:28:58.069116 4954 tablet_service.cc:1467] Processing CreateTablet for tablet 0d90eb9269884933a9af575b5af8a47e (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.069487 5277 consensus_queue.cc:237] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.070446 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d90eb9269884933a9af575b5af8a47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:58.071015 5219 consensus_queue.cc:260] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:58.075093 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:58.076467 5219 ts_tablet_manager.cc:1428] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260927 23:28:58.077389 5219 tablet_bootstrap.cc:492] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:58.080859 5221 ts_tablet_manager.cc:1428] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.136s user 0.008s sys 0.029s
I20260927 23:28:58.081919 5221 tablet_bootstrap.cc:492] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:58.095831 5054 leader_election.cc:302] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:58.097082 5329 raft_consensus.cc:2781] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:58.101683 5329 raft_consensus.cc:686] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:28:58.102715 5329 consensus_queue.cc:237] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:58.115188 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
W20260927 23:28:58.117224 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:28:58.120594 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
I20260927 23:28:58.122668 5221 tablet_bootstrap.cc:654] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:58.123044 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
W20260927 23:28:58.123371 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:28:58.123632 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:28:58.123864 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
I20260927 23:28:58.123728 5096 tablet_service.cc:1467] Processing CreateTablet for tablet 0d90eb9269884933a9af575b5af8a47e (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.124939 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d90eb9269884933a9af575b5af8a47e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:58.138003 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 77734e994d5d40dfb3efca344d396956 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:58.151342 5219 tablet_bootstrap.cc:654] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:58.152573 5229 tablet_bootstrap.cc:654] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:58.168795 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:28:58.169047 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:28:58.169195 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
W20260927 23:28:58.169346 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:28:58.169490 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
W20260927 23:28:58.174382 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
I20260927 23:28:58.169836 5223 ts_tablet_manager.cc:1918] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:58.187307 5223 log.cc:1196] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/c5552715138046279f00dcf497d05a72
I20260927 23:28:58.188762 5223 ts_tablet_manager.cc:1939] T c5552715138046279f00dcf497d05a72 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:28:58.191851 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet c5552715138046279f00dcf497d05a72 (table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]) successfully deleted
I20260927 23:28:58.220479 5224 ts_tablet_manager.cc:1918] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:58.220898 5224 log.cc:1196] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/77734e994d5d40dfb3efca344d396956
I20260927 23:28:58.226963 5224 ts_tablet_manager.cc:1939] T 77734e994d5d40dfb3efca344d396956 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:58.229394 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 77734e994d5d40dfb3efca344d396956 (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
I20260927 23:28:58.241736 5353 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:58.242236 5353 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
W20260927 23:28:58.247231 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:28:58.248144 5224 tablet_replica.cc:330] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:58.249855 5353 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:58.256768 5221 tablet_bootstrap.cc:492] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:58.257311 5221 ts_tablet_manager.cc:1397] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.176s user 0.009s sys 0.005s
I20260927 23:28:58.259438 5229 tablet_bootstrap.cc:492] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:58.259951 5229 ts_tablet_manager.cc:1397] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.196s user 0.005s sys 0.008s
I20260927 23:28:58.262524 5224 raft_consensus.cc:2229] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:58.264401 5229 raft_consensus.cc:348] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:58.267062 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
I20260927 23:28:58.268112 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 64cfbad1561a41048009b5fc819cc5de from {username='slave'} at 127.0.0.1:35136
W20260927 23:28:58.291839 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:28:58.295977 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 64cfbad1561a41048009b5fc819cc5de
I20260927 23:28:58.297616 5221 raft_consensus.cc:348] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.298337 5221 raft_consensus.cc:374] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:58.298568 5221 raft_consensus.cc:729] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:58.299278 5221 consensus_queue.cc:260] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.303900 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 64cfbad1561a41048009b5fc819cc5de with delete_type TABLET_DATA_DELETED (replaced by tablet 48dcf44bb7b744b9a98626dbfb6f4e6c) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:58.306216 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 64cfbad1561a41048009b5fc819cc5de deleted and replaced by tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:28:58.307317 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 64cfbad1561a41048009b5fc819cc5de because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 64cfbad1561a41048009b5fc819cc5de already in progress: creating tablet
I20260927 23:28:58.309289 5221 ts_tablet_manager.cc:1428] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.052s user 0.005s sys 0.000s
I20260927 23:28:58.313948 5221 tablet_bootstrap.cc:492] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:58.314743 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
W20260927 23:28:58.315064 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:28:58.315301 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:28:58.319818 5221 tablet_bootstrap.cc:654] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:58.321969 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
I20260927 23:28:58.301898 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 64cfbad1561a41048009b5fc819cc5de with delete_type TABLET_DATA_DELETED (replaced by tablet 48dcf44bb7b744b9a98626dbfb6f4e6c) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:58.324421 5224 raft_consensus.cc:2258] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:58.326176 5219 tablet_bootstrap.cc:492] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:58.326567 5219 ts_tablet_manager.cc:1397] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.249s user 0.010s sys 0.000s
I20260927 23:28:58.326920 5224 ts_tablet_manager.cc:1905] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:28:58.271127 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:28:58.333832 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
W20260927 23:28:58.334743 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 63653.3us
I20260927 23:28:58.334802 5329 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:58.271539 5310 raft_consensus.cc:482] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:58.336359 5310 raft_consensus.cc:504] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.338249 5310 leader_election.cc:288] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:58.339339 5096 tablet_service.cc:1514] Processing DeleteTablet for tablet 64cfbad1561a41048009b5fc819cc5de with delete_type TABLET_DATA_DELETED (replaced by tablet 48dcf44bb7b744b9a98626dbfb6f4e6c) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:58.347167 5051 leader_election.cc:341] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a483b5e32d55406bb3342af93f755028
I20260927 23:28:58.349401 4973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a483b5e32d55406bb3342af93f755028" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:58.350117 4973 raft_consensus.cc:2454] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
W20260927 23:28:58.351152 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 64cfbad1561a41048009b5fc819cc5de because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 64cfbad1561a41048009b5fc819cc5de already in progress: creating tablet
I20260927 23:28:58.352224 5054 leader_election.cc:302] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:58.353392 5310 raft_consensus.cc:2781] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:58.353727 5310 raft_consensus.cc:482] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:58.354024 5310 raft_consensus.cc:3037] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:58.358319 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d90eb9269884933a9af575b5af8a47e from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:58.358665 5367 raft_consensus.cc:482] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:58.359191 5367 raft_consensus.cc:504] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:58.360328 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
W20260927 23:28:58.360620 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:58.333832 5229 raft_consensus.cc:374] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:58.361312 5367 leader_election.cc:288] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:58.361402 5229 raft_consensus.cc:729] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:58.363163 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260ab7817da1431ebd80ee7dd00c40b4" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:58.363543 5229 consensus_queue.cc:260] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:58.364970 4910 leader_election.cc:341] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:58.365602 5229 ts_tablet_manager.cc:1428] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.105s user 0.005s sys 0.000s
I20260927 23:28:58.362545 5310 raft_consensus.cc:504] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:58.371735 5051 leader_election.cc:341] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a483b5e32d55406bb3342af93f755028
I20260927 23:28:58.368441 4833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260ab7817da1431ebd80ee7dd00c40b4" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:28:58.374752 4833 raft_consensus.cc:2454] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:28:58.377561 4907 leader_election.cc:302] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters: f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:58.378693 5367 raft_consensus.cc:2781] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:58.379019 5367 raft_consensus.cc:482] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:58.379297 5367 raft_consensus.cc:3037] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:58.387352 5329 consensus_queue.cc:1035] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:58.388702 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 0d90eb9269884933a9af575b5af8a47e
I20260927 23:28:58.392035 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d90eb9269884933a9af575b5af8a47e with delete_type TABLET_DATA_DELETED (replaced by tablet 5073ab2faa0042569b11193952cce2ac) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:58.393229 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d90eb9269884933a9af575b5af8a47e with delete_type TABLET_DATA_DELETED (replaced by tablet 5073ab2faa0042569b11193952cce2ac) from {username='slave'} at 127.0.0.1:53202
W20260927 23:28:58.393841 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 0d90eb9269884933a9af575b5af8a47e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0d90eb9269884933a9af575b5af8a47e already in progress: creating tablet
I20260927 23:28:58.394418 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 9dffe7e40ecb40c68a06f0cf43861777 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.395720 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dffe7e40ecb40c68a06f0cf43861777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:58.396291 4953 tablet_service.cc:1467] Processing CreateTablet for tablet 9dffe7e40ecb40c68a06f0cf43861777 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.397351 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dffe7e40ecb40c68a06f0cf43861777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:58.398222 5367 raft_consensus.cc:504] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.400841 4833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260ab7817da1431ebd80ee7dd00c40b4" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:28:58.401446 4833 raft_consensus.cc:3037] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:58.406100 5096 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d90eb9269884933a9af575b5af8a47e with delete_type TABLET_DATA_DELETED (replaced by tablet 5073ab2faa0042569b11193952cce2ac) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:58.406824 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 0d90eb9269884933a9af575b5af8a47e deleted and replaced by tablet 5073ab2faa0042569b11193952cce2ac
I20260927 23:28:58.409613 4833 raft_consensus.cc:2454] T 260ab7817da1431ebd80ee7dd00c40b4 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:28:58.416208 5096 tablet_service.cc:1467] Processing CreateTablet for tablet 9dffe7e40ecb40c68a06f0cf43861777 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.416970 5367 leader_election.cc:288] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:58.420351 4967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a483b5e32d55406bb3342af93f755028" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:28:58.421059 4967 raft_consensus.cc:3037] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:58.421010 4907 leader_election.cc:302] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:28:58.333637 5219 raft_consensus.cc:348] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.422009 5219 raft_consensus.cc:374] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:58.422303 5219 raft_consensus.cc:729] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:58.422971 5219 consensus_queue.cc:260] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.425064 5310 leader_election.cc:288] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:58.426064 5367 raft_consensus.cc:2781] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:58.428041 4967 raft_consensus.cc:2454] T a483b5e32d55406bb3342af93f755028 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:28:58.429337 5054 leader_election.cc:302] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:28:58.429337 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260ab7817da1431ebd80ee7dd00c40b4" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
W20260927 23:28:58.430954 4910 leader_election.cc:341] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:58.434392 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9dffe7e40ecb40c68a06f0cf43861777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:58.437251 5221 tablet_bootstrap.cc:492] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:58.437738 5221 ts_tablet_manager.cc:1397] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.124s user 0.008s sys 0.010s
W20260927 23:28:58.445822 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
I20260927 23:28:58.459434 5329 raft_consensus.cc:2781] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:58.460142 5329 raft_consensus.cc:686] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:28:58.460896 5329 consensus_queue.cc:237] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:58.480955 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 0d90eb9269884933a9af575b5af8a47e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0d90eb9269884933a9af575b5af8a47e already in progress: creating tablet
I20260927 23:28:58.486199 5296 raft_consensus.cc:482] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:58.486624 5296 raft_consensus.cc:504] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.486917 5221 raft_consensus.cc:348] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.487532 5221 raft_consensus.cc:374] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:58.487776 5221 raft_consensus.cc:729] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
W20260927 23:28:58.489109 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:58.489631 5296 leader_election.cc:288] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:58.489589 5221 consensus_queue.cc:260] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.491650 5221 ts_tablet_manager.cc:1428] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.054s user 0.006s sys 0.000s
I20260927 23:28:58.492939 5221 tablet_bootstrap.cc:492] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:58.493573 4907 leader_election.cc:341] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069
I20260927 23:28:58.497839 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c3c55f368dd4ba991daded55d2e3069" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:58.499604 4910 leader_election.cc:341] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:58.500018 4910 leader_election.cc:302] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:58.501276 5367 raft_consensus.cc:686] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
I20260927 23:28:58.501431 5296 raft_consensus.cc:2726] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:58.482273 5229 tablet_bootstrap.cc:492] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
W20260927 23:28:58.510105 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:58.526845 5221 tablet_bootstrap.cc:654] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:58.533640 5219 ts_tablet_manager.cc:1428] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.207s user 0.007s sys 0.000s
I20260927 23:28:58.548532 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 9dffe7e40ecb40c68a06f0cf43861777 from {username='slave'} at 127.0.0.1:35136
W20260927 23:28:58.551863 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
I20260927 23:28:58.554894 5367 consensus_queue.cc:237] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:58.563827 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:28:58.564231 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:58.564072 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 48dcf44bb7b744b9a98626dbfb6f4e6c (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.565217 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48dcf44bb7b744b9a98626dbfb6f4e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:58.574034 5224 ts_tablet_manager.cc:1918] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:58.574373 5224 log.cc:1196] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/c5552715138046279f00dcf497d05a72
I20260927 23:28:58.575158 5224 ts_tablet_manager.cc:1939] T c5552715138046279f00dcf497d05a72 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:58.577447 4953 tablet_service.cc:1467] Processing CreateTablet for tablet 48dcf44bb7b744b9a98626dbfb6f4e6c (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.578853 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48dcf44bb7b744b9a98626dbfb6f4e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:58.580003 5231 raft_consensus.cc:482] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:58.580525 5231 raft_consensus.cc:504] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.581110 5219 tablet_bootstrap.cc:492] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:58.582284 5231 leader_election.cc:288] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:58.584652 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64cfbad1561a41048009b5fc819cc5de" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:58.586359 4769 leader_election.cc:341] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:58.588752 5229 tablet_bootstrap.cc:654] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:58.593124 4615 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ac88a4e4bee64586bef7b6b409a63f4c consensus state has no leader
I20260927 23:28:58.594053 4967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64cfbad1561a41048009b5fc819cc5de" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:28:58.595664 4769 leader_election.cc:341] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:58.596068 4769 leader_election.cc:302] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:58.597210 5231 raft_consensus.cc:2726] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:58.611569 5219 tablet_bootstrap.cc:654] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:58.614470 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet c5552715138046279f00dcf497d05a72 (table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]) successfully deleted
I20260927 23:28:58.615525 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d90eb9269884933a9af575b5af8a47e from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:58.616888 5224 tablet_replica.cc:330] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:58.617569 5224 raft_consensus.cc:2229] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:58.617821 5396 raft_consensus.cc:482] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:58.618063 5224 raft_consensus.cc:2258] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:58.618314 5396 raft_consensus.cc:504] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.620810 5224 ts_tablet_manager.cc:1905] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:58.621577 5096 tablet_service.cc:1467] Processing CreateTablet for tablet 48dcf44bb7b744b9a98626dbfb6f4e6c (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.622776 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48dcf44bb7b744b9a98626dbfb6f4e6c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:58.622711 4907 leader_election.cc:341] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d
I20260927 23:28:58.623327 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2826716e5b184cc2b60d556ca2b89f4d" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:58.623463 5396 leader_election.cc:288] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
W20260927 23:28:58.624644 4910 leader_election.cc:341] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:58.625008 4910 leader_election.cc:302] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:58.625140 5221 tablet_bootstrap.cc:492] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:58.625662 5221 ts_tablet_manager.cc:1397] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.133s user 0.008s sys 0.004s
I20260927 23:28:58.627955 5221 raft_consensus.cc:348] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.628773 5396 raft_consensus.cc:2726] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:58.630251 5221 raft_consensus.cc:374] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:58.638718 5094 tablet_service.cc:1467] Processing CreateTablet for tablet 5073ab2faa0042569b11193952cce2ac (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.639770 5094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5073ab2faa0042569b11193952cce2ac. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:58.659729 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
W20260927 23:28:58.659997 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
W20260927 23:28:58.660156 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
I20260927 23:28:58.680639 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 9dffe7e40ecb40c68a06f0cf43861777
I20260927 23:28:58.683203 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 9dffe7e40ecb40c68a06f0cf43861777 deleted and replaced by tablet ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:28:58.685302 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dffe7e40ecb40c68a06f0cf43861777 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd4fd7dc4b14265b0eafe894b748131) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:58.686971 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 9dffe7e40ecb40c68a06f0cf43861777 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9dffe7e40ecb40c68a06f0cf43861777 already in progress: creating tablet
I20260927 23:28:58.689900 5229 tablet_bootstrap.cc:492] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:58.690316 5229 ts_tablet_manager.cc:1397] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.208s user 0.005s sys 0.003s
I20260927 23:28:58.635829 4812 tablet_service.cc:1467] Processing CreateTablet for tablet 5073ab2faa0042569b11193952cce2ac (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.682662 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dffe7e40ecb40c68a06f0cf43861777 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd4fd7dc4b14265b0eafe894b748131) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:58.697257 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5073ab2faa0042569b11193952cce2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:58.637656 4954 tablet_service.cc:1467] Processing CreateTablet for tablet 5073ab2faa0042569b11193952cce2ac (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.736474 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5073ab2faa0042569b11193952cce2ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:58.683935 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet 9dffe7e40ecb40c68a06f0cf43861777 with delete_type TABLET_DATA_DELETED (replaced by tablet ffd4fd7dc4b14265b0eafe894b748131) from {username='slave'} at 127.0.0.1:34620
W20260927 23:28:58.742337 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 9dffe7e40ecb40c68a06f0cf43861777 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9dffe7e40ecb40c68a06f0cf43861777 already in progress: creating tablet
I20260927 23:28:58.630565 5221 raft_consensus.cc:729] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:58.749881 5221 consensus_queue.cc:260] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.752051 4810 tablet_service.cc:1467] Processing CreateTablet for tablet ffd4fd7dc4b14265b0eafe894b748131 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.753291 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd4fd7dc4b14265b0eafe894b748131. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:58.760107 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
I20260927 23:28:58.776386 5096 tablet_service.cc:1467] Processing CreateTablet for tablet ffd4fd7dc4b14265b0eafe894b748131 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.778769 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd4fd7dc4b14265b0eafe894b748131. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:58.783365 4954 tablet_service.cc:1467] Processing CreateTablet for tablet ffd4fd7dc4b14265b0eafe894b748131 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.784426 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffd4fd7dc4b14265b0eafe894b748131. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:58.810467 4616 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.62:35155, user_credentials={real_user=slave}} blocked reactor thread for 95667.6us
I20260927 23:28:58.812878 5229 raft_consensus.cc:348] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.813819 5229 raft_consensus.cc:374] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:58.814110 5229 raft_consensus.cc:729] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:58.814841 5229 consensus_queue.cc:260] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.826985 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 9dffe7e40ecb40c68a06f0cf43861777 from {username='slave'} at 127.0.0.1:35136
W20260927 23:28:58.829506 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
I20260927 23:28:58.848815 5219 tablet_bootstrap.cc:492] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:58.849352 5219 ts_tablet_manager.cc:1397] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.268s user 0.003s sys 0.008s
I20260927 23:28:58.857263 5221 ts_tablet_manager.cc:1428] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.231s user 0.006s sys 0.000s
W20260927 23:28:58.862107 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
I20260927 23:28:58.865909 5329 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:58.866396 5329 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
W20260927 23:28:58.871181 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:28:58.872422 5329 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
I20260927 23:28:58.874595 5231 raft_consensus.cc:482] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:58.875133 5231 raft_consensus.cc:504] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:58.877121 5231 leader_election.cc:288] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:58.879503 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6846550519ce44fdafbc895ddca98d8b" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:58.880993 4769 leader_election.cc:341] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:58.882627 4967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6846550519ce44fdafbc895ddca98d8b" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:28:58.884155 4769 leader_election.cc:341] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:58.884588 4769 leader_election.cc:302] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:58.885929 5231 raft_consensus.cc:2726] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:58.887204 5329 consensus_queue.cc:1035] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:58.894647 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 48dcf44bb7b744b9a98626dbfb6f4e6c from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:58.899047 5367 raft_consensus.cc:482] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:58.899660 5367 raft_consensus.cc:504] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:58.903463 4907 leader_election.cc:341] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d
I20260927 23:28:58.905915 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4deca666d4e4d25aae3ed6f13e97c4d" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:58.907326 4910 leader_election.cc:341] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:58.907721 4910 leader_election.cc:302] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:58.908553 5396 raft_consensus.cc:2726] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:58.911566 5367 leader_election.cc:288] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:58.923583 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
I20260927 23:28:58.928107 5329 consensus_queue.cc:1035] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:28:58.932111 5096 tablet_service.cc:1514] Processing DeleteTablet for tablet 48dcf44bb7b744b9a98626dbfb6f4e6c with delete_type TABLET_DATA_DELETED (replaced by tablet e56378a5d42e47d395c881a0898da495) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:58.933036 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet 48dcf44bb7b744b9a98626dbfb6f4e6c with delete_type TABLET_DATA_DELETED (replaced by tablet e56378a5d42e47d395c881a0898da495) from {username='slave'} at 127.0.0.1:34620
W20260927 23:28:58.933911 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 48dcf44bb7b744b9a98626dbfb6f4e6c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48dcf44bb7b744b9a98626dbfb6f4e6c already in progress: creating tablet
W20260927 23:28:58.934563 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 48dcf44bb7b744b9a98626dbfb6f4e6c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48dcf44bb7b744b9a98626dbfb6f4e6c already in progress: creating tablet
I20260927 23:28:58.935690 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 48dcf44bb7b744b9a98626dbfb6f4e6c deleted and replaced by tablet e56378a5d42e47d395c881a0898da495
W20260927 23:28:58.947440 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:28:58.948093 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:28:58.950867 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 48dcf44bb7b744b9a98626dbfb6f4e6c with delete_type TABLET_DATA_DELETED (replaced by tablet e56378a5d42e47d395c881a0898da495) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:58.957779 5221 tablet_bootstrap.cc:492] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:58.986316 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
I20260927 23:28:58.992442 5329 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:58.993489 5051 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:40571, user_credentials={real_user=slave}} blocked reactor thread for 46108.6us
W20260927 23:28:58.994704 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:58.995683 4812 tablet_service.cc:1467] Processing CreateTablet for tablet e56378a5d42e47d395c881a0898da495 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:58.996780 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e56378a5d42e47d395c881a0898da495. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:58.997479 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:59.003113 4952 tablet_service.cc:1467] Processing CreateTablet for tablet e56378a5d42e47d395c881a0898da495 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:59.004248 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e56378a5d42e47d395c881a0898da495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:59.005791 5096 tablet_service.cc:1467] Processing CreateTablet for tablet e56378a5d42e47d395c881a0898da495 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:59.007009 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e56378a5d42e47d395c881a0898da495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:59.012641 5367 raft_consensus.cc:482] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:59.013214 5367 raft_consensus.cc:504] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:59.017143 4907 leader_election.cc:341] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88
I20260927 23:28:59.027328 5310 raft_consensus.cc:482] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:59.027825 5310 raft_consensus.cc:504] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.029747 5310 leader_election.cc:288] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:59.039455 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 64cfbad1561a41048009b5fc819cc5de from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:59.043053 4968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360070544fca4f62b0aac16d5889e1bc" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:28:59.043886 4968 raft_consensus.cc:2454] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:28:59.045068 5054 leader_election.cc:302] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:28:59.046162 5310 raft_consensus.cc:2781] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:28:59.046459 5310 raft_consensus.cc:482] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:28:59.046730 5310 raft_consensus.cc:3037] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:28:59.051712 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
I20260927 23:28:59.056134 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70cb3313148a47f8b6c692222508eb88" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:59.057739 4910 leader_election.cc:341] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:59.058246 4910 leader_election.cc:302] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:59.059123 5396 raft_consensus.cc:2726] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:59.063167 5367 leader_election.cc:288] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:59.072811 5229 ts_tablet_manager.cc:1428] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.382s user 0.004s sys 0.004s
W20260927 23:28:59.093770 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
I20260927 23:28:59.101722 5229 tablet_bootstrap.cc:492] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:59.108572 5329 consensus_queue.cc:1035] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:58.851753 5219 raft_consensus.cc:348] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.116030 5219 raft_consensus.cc:374] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.116421 5219 raft_consensus.cc:729] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
W20260927 23:28:59.149950 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:28:59.155923 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet bdecd389ff0341f4ab867eb160786ea9 from {username='slave'} at 127.0.0.1:35136
W20260927 23:28:54.553349 5256 debug-util.cc:398] Leaking SignalData structure 0x7b080001e820 after lost signal to thread 5209
W20260927 23:28:59.172716 5256 kernel_stack_watchdog.cc:198] Thread 5209 stuck at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/thread.cc:637 for 377ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20260927 23:28:53.180898 5257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:28:59.175592 5260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 23:28:59.179380 5259 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 6000 milliseconds
W20260927 23:28:59.179533 5209 thread.cc:636] OpenStack (cloud detector) Time spent creating pthread: real 6.006s user 0.518s sys 0.871s
W20260927 23:28:59.179888 5209 thread.cc:602] OpenStack (cloud detector) Time spent starting thread: real 6.006s user 0.518s sys 0.871s
I20260927 23:28:59.180161 5209 server_base.cc:1029] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
W20260927 23:28:59.185884 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
I20260927 23:28:59.117240 5219 consensus_queue.cc:260] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.194339 5219 ts_tablet_manager.cc:1428] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.345s user 0.000s sys 0.004s
I20260927 23:28:59.195212 5219 tablet_bootstrap.cc:492] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:59.087610 5310 raft_consensus.cc:504] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.199572 5310 leader_election.cc:288] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:28:59.199920 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
I20260927 23:28:59.201776 4967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "360070544fca4f62b0aac16d5889e1bc" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:28:59.203001 5209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 23:28:59.202363 4967 raft_consensus.cc:3037] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:28:59.218173 4967 raft_consensus.cc:2454] T 360070544fca4f62b0aac16d5889e1bc P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:28:59.219440 5054 leader_election.cc:302] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:28:59.220795 5221 tablet_bootstrap.cc:654] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:59.220983 5329 raft_consensus.cc:2781] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:28:59.221436 5329 raft_consensus.cc:686] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:28:59.222225 5329 consensus_queue.cc:237] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.237476 5229 tablet_bootstrap.cc:654] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:59.240638 5209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
W20260927 23:28:59.241345 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
W20260927 23:28:59.241627 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
W20260927 23:28:59.241804 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
I20260927 23:28:59.232142 5224 ts_tablet_manager.cc:1918] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:59.243201 5224 log.cc:1196] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/64cfbad1561a41048009b5fc819cc5de
I20260927 23:28:59.243399 5221 tablet_bootstrap.cc:492] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:59.243820 5224 ts_tablet_manager.cc:1939] T 64cfbad1561a41048009b5fc819cc5de P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:59.243835 5221 ts_tablet_manager.cc:1397] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.287s user 0.008s sys 0.001s
I20260927 23:28:59.246222 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 64cfbad1561a41048009b5fc819cc5de (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
W20260927 23:28:59.246587 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:28:59.246202 5221 raft_consensus.cc:348] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.246883 5221 raft_consensus.cc:374] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.247121 5221 raft_consensus.cc:729] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:59.236121 5219 tablet_bootstrap.cc:654] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:59.252449 5209 hybrid_clock.cc:648] HybridClock initialized: now 1790551739252311 us; error 130 us; skew 500 ppm
I20260927 23:28:59.253803 5209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260927 23:28:59.258658 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:28:59.261348 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet 0d90eb9269884933a9af575b5af8a47e because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 0d90eb9269884933a9af575b5af8a47e already in progress: creating tablet
I20260927 23:28:59.264479 5229 tablet_bootstrap.cc:492] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:59.264928 5229 ts_tablet_manager.cc:1397] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.163s user 0.008s sys 0.001s
I20260927 23:28:59.260166 5221 consensus_queue.cc:260] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.267143 5229 raft_consensus.cc:348] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.268738 5221 ts_tablet_manager.cc:1428] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.025s user 0.002s sys 0.006s
W20260927 23:28:59.271898 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet 9dffe7e40ecb40c68a06f0cf43861777 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 9dffe7e40ecb40c68a06f0cf43861777 already in progress: creating tablet
W20260927 23:28:59.273644 4614 catalog_manager.cc:4821] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): delete failed for tablet 48dcf44bb7b744b9a98626dbfb6f4e6c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 48dcf44bb7b744b9a98626dbfb6f4e6c already in progress: creating tablet
I20260927 23:28:59.274042 5219 tablet_bootstrap.cc:492] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:59.267575 5353 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:59.274400 5219 ts_tablet_manager.cc:1397] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.079s user 0.008s sys 0.000s
I20260927 23:28:59.274547 5353 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:28:59.276871 5353 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
I20260927 23:28:59.276587 5219 raft_consensus.cc:348] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.277217 5219 raft_consensus.cc:374] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.277446 5219 raft_consensus.cc:729] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:59.278069 5219 consensus_queue.cc:260] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.278805 5229 raft_consensus.cc:374] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.279115 5229 raft_consensus.cc:729] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:59.280002 5219 ts_tablet_manager.cc:1428] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260927 23:28:59.280850 5219 tablet_bootstrap.cc:492] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:59.281503 5229 consensus_queue.cc:260] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.285485 5219 tablet_bootstrap.cc:654] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:59.292824 5229 ts_tablet_manager.cc:1428] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.028s user 0.004s sys 0.001s
I20260927 23:28:59.296985 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 5073ab2faa0042569b11193952cce2ac from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:59.304101 5229 tablet_bootstrap.cc:492] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:28:59.317952 5229 tablet_bootstrap.cc:654] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:59.323284 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 5073ab2faa0042569b11193952cce2ac
I20260927 23:28:59.325672 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 5073ab2faa0042569b11193952cce2ac with delete_type TABLET_DATA_DELETED (replaced by tablet df6ea36da9f6455bb399c2229dcb3790) from {username='slave'} at 127.0.0.1:34620
I20260927 23:28:59.326965 5219 tablet_bootstrap.cc:492] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:59.326908 5096 tablet_service.cc:1514] Processing DeleteTablet for tablet 5073ab2faa0042569b11193952cce2ac with delete_type TABLET_DATA_DELETED (replaced by tablet df6ea36da9f6455bb399c2229dcb3790) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:59.327381 5219 ts_tablet_manager.cc:1397] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.047s user 0.012s sys 0.029s
I20260927 23:28:59.328192 5229 tablet_bootstrap.cc:492] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:28:59.328598 5229 ts_tablet_manager.cc:1397] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.000s
I20260927 23:28:59.330781 5229 raft_consensus.cc:348] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.331426 5229 raft_consensus.cc:374] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.331666 5229 raft_consensus.cc:729] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:28:59.332229 5229 consensus_queue.cc:260] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.335505 4622 catalog_manager.cc:6565] ReplaceTablet: tablet 5073ab2faa0042569b11193952cce2ac deleted and replaced by tablet df6ea36da9f6455bb399c2229dcb3790
I20260927 23:28:59.337177 5219 raft_consensus.cc:348] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.337944 5219 raft_consensus.cc:374] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.338297 5219 raft_consensus.cc:729] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:59.339074 5219 consensus_queue.cc:260] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:59.344789 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
W20260927 23:28:59.333844 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 5073ab2faa0042569b11193952cce2ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5073ab2faa0042569b11193952cce2ac already in progress: creating tablet
I20260927 23:28:59.348429 5219 ts_tablet_manager.cc:1428] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.021s user 0.007s sys 0.001s
W20260927 23:28:59.350569 4910 consensus_peers.cc:487] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d -> Peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Couldn't send request to peer f5ec98b2972d44838fa273eb7a10d01a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:59.354637 5219 tablet_bootstrap.cc:492] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:59.361625 5219 tablet_bootstrap.cc:654] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:59.376616 5219 tablet_bootstrap.cc:492] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:59.377099 5219 ts_tablet_manager.cc:1397] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.023s user 0.012s sys 0.004s
W20260927 23:28:59.384009 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
I20260927 23:28:59.401448 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 6846550519ce44fdafbc895ddca98d8b from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:59.400810 5219 raft_consensus.cc:348] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.402491 5219 raft_consensus.cc:374] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.405115 5219 raft_consensus.cc:729] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:59.405786 5219 consensus_queue.cc:260] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.421702 5219 ts_tablet_manager.cc:1428] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.044s user 0.005s sys 0.001s
I20260927 23:28:59.422726 5219 tablet_bootstrap.cc:492] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:59.445894 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 6846550519ce44fdafbc895ddca98d8b
I20260927 23:28:59.445859 4953 tablet_service.cc:1467] Processing CreateTablet for tablet df6ea36da9f6455bb399c2229dcb3790 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:59.446954 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df6ea36da9f6455bb399c2229dcb3790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:59.449035 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 6846550519ce44fdafbc895ddca98d8b deleted and replaced by tablet 309465f419c24549bf5416a4503b9299
I20260927 23:28:59.453150 4951 tablet_service.cc:1514] Processing DeleteTablet for tablet 6846550519ce44fdafbc895ddca98d8b with delete_type TABLET_DATA_DELETED (replaced by tablet 309465f419c24549bf5416a4503b9299) from {username='slave'} at 127.0.0.1:34620
W20260927 23:28:59.453538 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
W20260927 23:28:59.455974 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
I20260927 23:28:59.465265 5221 tablet_bootstrap.cc:492] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:59.465675 5219 tablet_bootstrap.cc:654] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:59.470587 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
W20260927 23:28:59.469332 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 5073ab2faa0042569b11193952cce2ac because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 5073ab2faa0042569b11193952cce2ac already in progress: creating tablet
W20260927 23:28:59.471480 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 6846550519ce44fdafbc895ddca98d8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6846550519ce44fdafbc895ddca98d8b already in progress: creating tablet
I20260927 23:28:59.471817 5221 tablet_bootstrap.cc:654] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:59.472712 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:28:59.480389 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet c5552715138046279f00dcf497d05a72
W20260927 23:28:59.482084 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 63721.3us
I20260927 23:28:59.483875 5219 tablet_bootstrap.cc:492] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:59.484467 5219 ts_tablet_manager.cc:1397] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.062s user 0.007s sys 0.004s
I20260927 23:28:59.486848 5219 raft_consensus.cc:348] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.487557 5219 raft_consensus.cc:374] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.487955 5219 raft_consensus.cc:729] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
W20260927 23:28:59.487207 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:28:59.492585 5096 tablet_service.cc:1514] Processing DeleteTablet for tablet 6846550519ce44fdafbc895ddca98d8b with delete_type TABLET_DATA_DELETED (replaced by tablet 309465f419c24549bf5416a4503b9299) from {username='slave'} at 127.0.0.1:58244
I20260927 23:28:59.492113 5219 consensus_queue.cc:260] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.503253 5219 ts_tablet_manager.cc:1428] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260927 23:28:59.504473 5219 tablet_bootstrap.cc:492] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:59.506954 5231 raft_consensus.cc:482] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:59.507440 5231 raft_consensus.cc:504] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.509505 5231 leader_election.cc:288] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:59.511646 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdecd389ff0341f4ab867eb160786ea9" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:59.513387 4769 leader_election.cc:341] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:59.515738 5219 tablet_bootstrap.cc:654] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:59.516408 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdecd389ff0341f4ab867eb160786ea9" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:28:59.518427 4769 leader_election.cc:341] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:59.519063 4769 leader_election.cc:302] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:59.520071 5231 raft_consensus.cc:2726] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:59.531476 5310 consensus_queue.cc:1035] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:28:59.536940 5219 tablet_bootstrap.cc:492] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:59.537433 5219 ts_tablet_manager.cc:1397] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.033s user 0.012s sys 0.000s
I20260927 23:28:59.539743 5219 raft_consensus.cc:348] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.540488 5219 raft_consensus.cc:374] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.540735 5219 raft_consensus.cc:729] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:59.541291 5219 consensus_queue.cc:260] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.544763 5219 ts_tablet_manager.cc:1428] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260927 23:28:59.546388 5219 tablet_bootstrap.cc:492] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
W20260927 23:28:59.481567 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
I20260927 23:28:59.570037 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 9dffe7e40ecb40c68a06f0cf43861777 from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:59.571259 5231 raft_consensus.cc:482] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:59.571756 5231 raft_consensus.cc:504] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.574609 5231 leader_election.cc:288] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
W20260927 23:28:59.495978 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 6846550519ce44fdafbc895ddca98d8b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6846550519ce44fdafbc895ddca98d8b already in progress: creating tablet
W20260927 23:28:59.576663 4617 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.2.57.3:45519, user_credentials={real_user=slave}} blocked reactor thread for 80724us
I20260927 23:28:59.579305 5096 tablet_service.cc:1467] Processing CreateTablet for tablet df6ea36da9f6455bb399c2229dcb3790 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:59.580458 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df6ea36da9f6455bb399c2229dcb3790. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:59.586066 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d90eb9269884933a9af575b5af8a47e" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:59.590730 5367 raft_consensus.cc:482] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:59.591257 5367 raft_consensus.cc:504] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.564146 5219 tablet_bootstrap.cc:654] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:28:59.592485 4769 leader_election.cc:341] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:59.593690 5367 leader_election.cc:288] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:59.605715 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d90eb9269884933a9af575b5af8a47e" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:28:59.607452 4769 leader_election.cc:341] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:59.607879 4769 leader_election.cc:302] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:59.608608 5231 raft_consensus.cc:2726] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:59.609405 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbdfdfc23b0c4e3d9224ac2815c6cc65" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:59.612118 4910 leader_election.cc:341] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:28:59.613801 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:59.614529 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:28:59.615154 5051 leader_election.cc:341] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc
I20260927 23:28:59.631788 5367 raft_consensus.cc:482] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:59.632356 5367 raft_consensus.cc:504] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.635957 5367 leader_election.cc:288] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
W20260927 23:28:59.648765 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
I20260927 23:28:59.654457 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8328db65eb43ebb79cd9d7a1ddb0d8" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:59.655028 5221 tablet_bootstrap.cc:492] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:59.655467 5221 ts_tablet_manager.cc:1397] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.190s user 0.018s sys 0.020s
W20260927 23:28:59.655762 4910 leader_election.cc:341] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:59.659595 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 6846550519ce44fdafbc895ddca98d8b from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:59.673089 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 5073ab2faa0042569b11193952cce2ac with delete_type TABLET_DATA_DELETED (replaced by tablet df6ea36da9f6455bb399c2229dcb3790) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:59.674096 5221 raft_consensus.cc:348] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.674808 5224 tablet_replica.cc:330] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:59.674840 5221 raft_consensus.cc:374] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.674949 4812 tablet_service.cc:1467] Processing CreateTablet for tablet df6ea36da9f6455bb399c2229dcb3790 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:59.675215 5221 raft_consensus.cc:729] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:59.675552 5224 raft_consensus.cc:2229] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:59.675999 5224 raft_consensus.cc:2258] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:59.676066 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df6ea36da9f6455bb399c2229dcb3790. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:28:59.677004 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
I20260927 23:28:59.680120 5219 tablet_bootstrap.cc:492] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:59.680521 5219 ts_tablet_manager.cc:1397] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.134s user 0.022s sys 0.022s
I20260927 23:28:59.682567 5219 raft_consensus.cc:348] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.683177 5219 raft_consensus.cc:374] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.683430 5219 raft_consensus.cc:729] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:28:59.675848 5221 consensus_queue.cc:260] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:59.693459 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
I20260927 23:28:59.687834 5224 ts_tablet_manager.cc:1905] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:59.689605 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet 6846550519ce44fdafbc895ddca98d8b with delete_type TABLET_DATA_DELETED (replaced by tablet 309465f419c24549bf5416a4503b9299) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:59.698556 4953 tablet_service.cc:1467] Processing CreateTablet for tablet 309465f419c24549bf5416a4503b9299 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:59.699597 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 309465f419c24549bf5416a4503b9299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:59.683992 5219 consensus_queue.cc:260] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.702749 5221 ts_tablet_manager.cc:1428] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.047s user 0.007s sys 0.000s
W20260927 23:28:59.711946 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
I20260927 23:28:59.711555 5221 tablet_bootstrap.cc:492] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:28:59.713711 5219 ts_tablet_manager.cc:1428] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.033s user 0.002s sys 0.003s
I20260927 23:28:59.716660 5367 raft_consensus.cc:482] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:59.717204 5367 raft_consensus.cc:504] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.719106 5367 leader_election.cc:288] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:28:59.729740 5224 ts_tablet_manager.cc:1918] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:59.730101 5224 log.cc:1196] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/5073ab2faa0042569b11193952cce2ac
I20260927 23:28:59.730672 5224 ts_tablet_manager.cc:1939] T 5073ab2faa0042569b11193952cce2ac P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:59.732824 5224 tablet_replica.cc:330] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:59.733466 5224 raft_consensus.cc:2229] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:59.733850 5224 raft_consensus.cc:2258] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:59.735282 5224 ts_tablet_manager.cc:1905] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:59.735316 5221 tablet_bootstrap.cc:654] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:59.740912 5231 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:59.741336 5231 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.743160 5231 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
W20260927 23:28:59.752182 4907 leader_election.cc:341] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65
I20260927 23:28:59.752679 4907 leader_election.cc:302] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:59.753857 5367 raft_consensus.cc:2726] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:59.754019 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd4fd7dc4b14265b0eafe894b748131" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:28:59.756083 4769 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:59.748198 5224 ts_tablet_manager.cc:1918] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:59.760686 5224 log.cc:1196] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/6846550519ce44fdafbc895ddca98d8b
I20260927 23:28:59.761309 5224 ts_tablet_manager.cc:1939] T 6846550519ce44fdafbc895ddca98d8b P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:28:59.765316 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 5073ab2faa0042569b11193952cce2ac (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
W20260927 23:28:59.767328 4907 leader_election.cc:341] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8
I20260927 23:28:59.767771 4907 leader_election.cc:302] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:59.768898 5367 raft_consensus.cc:2726] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:59.770020 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 6846550519ce44fdafbc895ddca98d8b (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
I20260927 23:28:59.773976 5367 consensus_queue.cc:1035] T 260ab7817da1431ebd80ee7dd00c40b4 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:28:59.776844 4907 leader_election.cc:341] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956
I20260927 23:28:59.777181 5221 tablet_bootstrap.cc:492] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:59.777598 5221 ts_tablet_manager.cc:1397] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.070s user 0.009s sys 0.001s
I20260927 23:28:59.778894 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 309465f419c24549bf5416a4503b9299 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:59.779969 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 309465f419c24549bf5416a4503b9299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:59.784646 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 48dcf44bb7b744b9a98626dbfb6f4e6c from {username='slave'} at 127.0.0.1:35136
I20260927 23:28:59.785832 5221 raft_consensus.cc:348] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.786522 5221 raft_consensus.cc:374] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.786777 5221 raft_consensus.cc:729] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:59.791816 5221 consensus_queue.cc:260] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.796738 5209 webserver.cc:460] Webserver started at http://127.2.57.62:34737/ using document root <none> and password file <none>
I20260927 23:28:59.799221 5209 fs_manager.cc:359] Metadata directory not provided
I20260927 23:28:59.801793 5221 ts_tablet_manager.cc:1428] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.024s user 0.007s sys 0.000s
W20260927 23:28:59.806437 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
W20260927 23:28:59.814172 5051 leader_election.cc:341] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc
I20260927 23:28:59.815668 5221 tablet_bootstrap.cc:492] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:28:59.816186 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
W20260927 23:28:59.825254 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:28:59.825991 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:28:59.829939 5221 tablet_bootstrap.cc:654] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:59.846191 5209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
W20260927 23:28:59.849330 4616 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.62:35155, user_credentials={real_user=slave}} blocked reactor thread for 44284.4us
I20260927 23:28:59.849715 5434 raft_consensus.cc:482] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:28:59.855062 4616 consensus_peers.cc:487] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d -> Peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Couldn't send request to peer d9ed71d97661411891f2d2de4cdc2269. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260927 23:28:59.864516 5219 tablet_bootstrap.cc:492] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:59.869406 5219 tablet_bootstrap.cc:654] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:59.880725 5219 tablet_bootstrap.cc:492] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:28:59.881204 5219 ts_tablet_manager.cc:1397] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.004s
I20260927 23:28:59.883314 5219 raft_consensus.cc:348] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:28:59.852599 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 198348us
W20260927 23:28:59.853582 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:28:59.858577 5437 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:59.863324 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77734e994d5d40dfb3efca344d396956" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:59.863221 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffd4fd7dc4b14265b0eafe894b748131" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:28:59.884625 5219 raft_consensus.cc:374] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.885076 5219 raft_consensus.cc:729] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
W20260927 23:28:59.887265 4910 leader_election.cc:341] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:59.887673 4910 leader_election.cc:302] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:59.885782 5219 consensus_queue.cc:260] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.889367 5367 raft_consensus.cc:2726] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:28:59.888839 4769 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:59.893348 4769 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:59.894479 5231 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:59.896548 5219 ts_tablet_manager.cc:1428] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.015s user 0.004s sys 0.001s
I20260927 23:28:59.906840 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet df6ea36da9f6455bb399c2229dcb3790 from {username='slave'} at 127.0.0.1:35136
W20260927 23:28:59.913683 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
I20260927 23:28:59.921666 5096 tablet_service.cc:1467] Processing CreateTablet for tablet 309465f419c24549bf5416a4503b9299 (DEFAULT_TABLE table=default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:59.924280 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 309465f419c24549bf5416a4503b9299. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:59.927772 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet df6ea36da9f6455bb399c2229dcb3790
I20260927 23:28:59.928944 4627 catalog_manager.cc:6565] ReplaceTablet: tablet df6ea36da9f6455bb399c2229dcb3790 deleted and replaced by tablet 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:28:59.931771 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet df6ea36da9f6455bb399c2229dcb3790 with delete_type TABLET_DATA_DELETED (replaced by tablet 327adcc2a27f42acb3a77d49fbf73796) from {username='slave'} at 127.0.0.1:34620
W20260927 23:28:59.932268 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
W20260927 23:28:59.932499 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
W20260927 23:28:59.932662 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
W20260927 23:28:59.933511 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet df6ea36da9f6455bb399c2229dcb3790 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df6ea36da9f6455bb399c2229dcb3790 already in progress: creating tablet
I20260927 23:28:59.938696 5231 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:59.939126 5231 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.940892 5231 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
W20260927 23:28:59.942062 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
I20260927 23:28:59.942456 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e56378a5d42e47d395c881a0898da495" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:28:59.943686 4769 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:59.945447 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet df6ea36da9f6455bb399c2229dcb3790 with delete_type TABLET_DATA_DELETED (replaced by tablet 327adcc2a27f42acb3a77d49fbf73796) from {username='slave'} at 127.0.0.1:53202
I20260927 23:28:59.946072 5224 tablet_replica.cc:330] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:28:59.946815 5224 raft_consensus.cc:2229] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:28:59.947229 5224 raft_consensus.cc:2258] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:28:59.947254 5440 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:28:59.949186 5224 ts_tablet_manager.cc:1905] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:28:59.947750 5440 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:28:59.950721 5219 tablet_bootstrap.cc:492] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:28:59.953473 5221 tablet_bootstrap.cc:492] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:28:59.953889 5221 ts_tablet_manager.cc:1397] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.138s user 0.017s sys 0.004s
I20260927 23:28:59.955452 5219 tablet_bootstrap.cc:654] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:28:59.956363 5221 raft_consensus.cc:348] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.956998 5221 raft_consensus.cc:374] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:28:59.957221 5221 raft_consensus.cc:729] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:28:59.957742 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e56378a5d42e47d395c881a0898da495" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:28:59.960891 4769 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:28:59.961311 4769 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:28:59.962623 5231 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:28:59.964882 5094 tablet_service.cc:1514] Processing DeleteTablet for tablet df6ea36da9f6455bb399c2229dcb3790 with delete_type TABLET_DATA_DELETED (replaced by tablet 327adcc2a27f42acb3a77d49fbf73796) from {username='slave'} at 127.0.0.1:58244
W20260927 23:28:59.967440 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet df6ea36da9f6455bb399c2229dcb3790 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet df6ea36da9f6455bb399c2229dcb3790 already in progress: creating tablet
I20260927 23:28:59.967645 5440 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:28:59.969785 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
I20260927 23:28:59.984925 4953 tablet_service.cc:1467] Processing CreateTablet for tablet 327adcc2a27f42acb3a77d49fbf73796 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:28:59.986251 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 327adcc2a27f42acb3a77d49fbf73796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:28:59.983160 5221 consensus_queue.cc:260] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:28:59.993040 5224 ts_tablet_manager.cc:1918] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:28:59.996975 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 48dcf44bb7b744b9a98626dbfb6f4e6c from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:00.000044 5219 tablet_bootstrap.cc:492] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:00.002045 5219 ts_tablet_manager.cc:1397] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.051s user 0.009s sys 0.007s
W20260927 23:29:00.003786 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:28:59.997532 5224 log.cc:1196] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/df6ea36da9f6455bb399c2229dcb3790
I20260927 23:29:00.004967 5224 ts_tablet_manager.cc:1939] T df6ea36da9f6455bb399c2229dcb3790 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:00.007817 5094 tablet_service.cc:1467] Processing CreateTablet for tablet 327adcc2a27f42acb3a77d49fbf73796 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:00.008975 5094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 327adcc2a27f42acb3a77d49fbf73796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:00.005558 5219 raft_consensus.cc:348] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.009325 5219 raft_consensus.cc:374] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.010743 5219 raft_consensus.cc:729] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.011423 5219 consensus_queue.cc:260] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.012547 5221 ts_tablet_manager.cc:1428] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.058s user 0.008s sys 0.000s
I20260927 23:29:00.012589 5231 raft_consensus.cc:482] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:00.012995 5231 raft_consensus.cc:504] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.014751 5231 leader_election.cc:288] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:00.015507 5423 consensus_queue.cc:1035] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260927 23:29:00.021198 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dffe7e40ecb40c68a06f0cf43861777" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:00.021268 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dffe7e40ecb40c68a06f0cf43861777" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:00.014369 5051 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:40571, user_credentials={real_user=slave}} blocked reactor thread for 115771us
W20260927 23:29:00.022375 4769 leader_election.cc:341] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:29:00.023000 4769 leader_election.cc:341] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.023347 4769 leader_election.cc:302] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:00.024219 5440 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:00.024690 5440 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.025780 5231 raft_consensus.cc:2726] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:00.026500 5440 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:28:59.897223 5434 raft_consensus.cc:504] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:00.052440 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:29:00.053589 5434 leader_election.cc:288] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:00.056182 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:29:00.056888 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2477e4ce30497fa94f2dfdb3c31a3d" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:00.057461 4970 raft_consensus.cc:2454] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:00.060984 5434 consensus_queue.cc:1035] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:00.062763 5054 leader_election.cc:302] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:00.068629 5441 raft_consensus.cc:2781] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:00.068988 5441 raft_consensus.cc:482] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:00.069280 5441 raft_consensus.cc:3037] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:00.073884 5221 tablet_bootstrap.cc:492] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:00.078099 5229 ts_tablet_manager.cc:1428] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.749s user 0.011s sys 0.006s
W20260927 23:29:00.078466 5229 ts_tablet_manager.cc:1460] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a: Tablet startup took 749ms
I20260927 23:29:00.079036 5441 raft_consensus.cc:504] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.080935 5221 tablet_bootstrap.cc:654] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.083637 5231 raft_consensus.cc:482] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:00.084134 5231 raft_consensus.cc:504] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.086160 5231 leader_election.cc:288] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:00.088442 5434 raft_consensus.cc:482] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:29:00.088958 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
I20260927 23:29:00.088734 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48dcf44bb7b744b9a98626dbfb6f4e6c" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:00.088887 5434 raft_consensus.cc:504] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:00.089969 4769 leader_election.cc:341] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.090811 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet df6ea36da9f6455bb399c2229dcb3790 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:29:00.091082 5434 leader_election.cc:288] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:00.106168 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:29:00.122726 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:29:00.123396 4973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260ab7817da1431ebd80ee7dd00c40b4" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:00.128552 4973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48dcf44bb7b744b9a98626dbfb6f4e6c" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:29:00.132118 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
W20260927 23:29:00.132490 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
W20260927 23:29:00.135308 4769 leader_election.cc:341] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.135743 4769 leader_election.cc:302] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:00.136219 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260ab7817da1431ebd80ee7dd00c40b4" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:00.136783 5231 raft_consensus.cc:2726] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:00.141511 5219 ts_tablet_manager.cc:1428] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.139s user 0.033s sys 0.006s
I20260927 23:29:00.150180 5221 tablet_bootstrap.cc:492] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:00.150671 5221 ts_tablet_manager.cc:1397] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.077s user 0.007s sys 0.016s
W20260927 23:29:00.153575 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:29:00.153852 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
I20260927 23:29:00.154861 5221 raft_consensus.cc:348] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:00.078739 5229 ts_tablet_manager.cc:1462] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a: Trace:
0927 23:28:55.629207 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:55.629370 (+ 163us) service_pool.cc:227] Handling call
0927 23:28:55.630899 (+ 1529us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0927 23:28:55.630986 (+ 87us) ts_tablet_manager.cc:676] Creating new metadata...
0927 23:28:55.666797 (+ 35811us) tablet_metadata.cc:641] Metadata flushed
0927 23:28:55.728565 (+ 61768us) tablet_replica.cc:205] Creating consensus instance
0927 23:28:55.729115 (+ 550us) inbound_call.cc:173] Queueing success response
0927 23:28:59.303796 (+3574681us) ts_tablet_manager.cc:1373] Bootstrapping tablet
0927 23:28:59.328839 (+ 25043us) ts_tablet_manager.cc:1429] Starting tablet replica
0927 23:28:59.329367 (+ 528us) tablet_replica.cc:259] Starting instrumentation
0927 23:28:59.329818 (+ 451us) tablet_replica.cc:278] Starting consensus
Metrics: {"mutex_wait_us":535079,"raft.queue_time_us":554502,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":279,"tablet-open.queue_time_us":3574800,"thread_start_us":208369,"threads_started":1}
I20260927 23:29:00.162532 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 9dffe7e40ecb40c68a06f0cf43861777 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:00.163358 5221 raft_consensus.cc:374] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.164063 5221 raft_consensus.cc:729] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
W20260927 23:29:00.164216 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
I20260927 23:29:00.172820 5221 consensus_queue.cc:260] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.181294 5221 ts_tablet_manager.cc:1428] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.030s user 0.005s sys 0.000s
I20260927 23:29:00.185266 5221 tablet_bootstrap.cc:492] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:29:00.186396 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
I20260927 23:29:00.186600 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 327adcc2a27f42acb3a77d49fbf73796 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:00.187883 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 327adcc2a27f42acb3a77d49fbf73796. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:00.207712 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet ffd4fd7dc4b14265b0eafe894b748131 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:00.214356 5221 tablet_bootstrap.cc:654] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:00.214453 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
I20260927 23:29:00.221930 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:29:00.227662 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd4fd7dc4b14265b0eafe894b748131 with delete_type TABLET_DATA_DELETED (replaced by tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1) from {username='slave'} at 127.0.0.1:34620
W20260927 23:29:00.228904 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:29:00.229036 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet ffd4fd7dc4b14265b0eafe894b748131 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffd4fd7dc4b14265b0eafe894b748131 already in progress: creating tablet
I20260927 23:29:00.229316 5051 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:00.229744 5117 raft_consensus.cc:3037] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:00.230278 5229 tablet_bootstrap.cc:492] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:00.230501 4627 catalog_manager.cc:6565] ReplaceTablet: tablet ffd4fd7dc4b14265b0eafe894b748131 deleted and replaced by tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:29:00.242038 5221 tablet_bootstrap.cc:492] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:00.248570 5221 ts_tablet_manager.cc:1397] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.063s user 0.009s sys 0.000s
I20260927 23:29:00.249677 5441 leader_election.cc:288] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:00.249884 5209 fs_manager.cc:709] Time spent opening directory manager: real 0.293s user 0.004s sys 0.003s
I20260927 23:29:00.252354 5221 raft_consensus.cc:348] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.254216 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd4fd7dc4b14265b0eafe894b748131 with delete_type TABLET_DATA_DELETED (replaced by tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1) from {username='slave'} at 127.0.0.1:53202
I20260927 23:29:00.254899 5224 tablet_replica.cc:330] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:00.255584 5224 raft_consensus.cc:2229] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:00.255972 5224 raft_consensus.cc:2258] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:00.257650 5224 ts_tablet_manager.cc:1905] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:00.259225 5229 tablet_bootstrap.cc:654] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.265177 5457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 23:29:00.272439 5221 raft_consensus.cc:374] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260927 23:29:00.273912 5051 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:40571, user_credentials={real_user=slave}} blocked reactor thread for 45043.3us
W20260927 23:29:00.274811 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:29:00.275202 5051 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
W20260927 23:29:00.276496 5051 leader_election.cc:341] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d
I20260927 23:29:00.276780 5427 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:00.276126 5219 tablet_bootstrap.cc:492] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:00.277348 5051 leader_election.cc:302] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:00.279505 5434 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:00.283330 5219 tablet_bootstrap.cc:654] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:00.283062 5051 leader_election.cc:341] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d
I20260927 23:29:00.283135 5441 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:00.284621 5441 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.286365 5441 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:00.287653 5209 fs_manager.cc:725] Time spent opening block manager: real 0.032s user 0.003s sys 0.001s
I20260927 23:29:00.288578 5221 raft_consensus.cc:729] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.288993 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:00.289958 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:29:00.289937 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:29:00.289713 5221 consensus_queue.cc:260] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:00.293008 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 149940us
W20260927 23:29:00.298244 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
I20260927 23:29:00.307339 5229 tablet_bootstrap.cc:492] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:00.307783 5229 ts_tablet_manager.cc:1397] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.078s user 0.006s sys 0.004s
I20260927 23:29:00.299953 4953 tablet_service.cc:1467] Processing CreateTablet for tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:00.288121 5209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data,/tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/wal
uuid: "d9ed71d97661411891f2d2de4cdc2269"
format_stamp: "Formatted at 2026-09-27 23:28:39 on dist-test-slave-jbx2"
I20260927 23:29:00.311595 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a2477e4ce30497fa94f2dfdb3c31a3d" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:00.312114 4970 raft_consensus.cc:3037] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:00.310878 5209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 23:29:00.313633 5221 ts_tablet_manager.cc:1428] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.063s user 0.005s sys 0.000s
I20260927 23:29:00.314713 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:29:00.319178 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
W20260927 23:29:00.319406 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
I20260927 23:29:00.328033 5094 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd4fd7dc4b14265b0eafe894b748131 with delete_type TABLET_DATA_DELETED (replaced by tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1) from {username='slave'} at 127.0.0.1:58244
I20260927 23:29:00.329116 4970 raft_consensus.cc:2454] T 7a2477e4ce30497fa94f2dfdb3c31a3d P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:00.329906 5096 tablet_service.cc:1467] Processing CreateTablet for tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1 (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:00.330871 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:00.330269 5221 tablet_bootstrap.cc:492] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:00.338882 5219 tablet_bootstrap.cc:492] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:00.339401 5219 ts_tablet_manager.cc:1397] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.063s user 0.011s sys 0.000s
I20260927 23:29:00.341660 5219 raft_consensus.cc:348] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.342403 5219 raft_consensus.cc:374] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.342677 5219 raft_consensus.cc:729] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.343312 5219 consensus_queue.cc:260] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.345616 5219 ts_tablet_manager.cc:1428] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260927 23:29:00.348593 4627 catalog_manager.cc:5587] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571) does not have the latest schema for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]). Expected version 1 got 0
W20260927 23:29:00.350148 4627 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 327adcc2a27f42acb3a77d49fbf73796 consensus state has no leader
W20260927 23:29:00.354836 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet ffd4fd7dc4b14265b0eafe894b748131 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet ffd4fd7dc4b14265b0eafe894b748131 already in progress: creating tablet
W20260927 23:29:00.355751 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:29:00.348608 5229 raft_consensus.cc:348] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.353024 5224 ts_tablet_manager.cc:1918] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:00.357918 5224 log.cc:1196] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:29:00.356931 5229 raft_consensus.cc:374] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.358376 5229 raft_consensus.cc:729] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.349123 5437 raft_consensus.cc:2726] T 260ab7817da1431ebd80ee7dd00c40b4 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:00.368450 5224 ts_tablet_manager.cc:1939] T ffd4fd7dc4b14265b0eafe894b748131 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:00.373160 5221 tablet_bootstrap.cc:654] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.378489 5229 consensus_queue.cc:260] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:00.382448 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 327adcc2a27f42acb3a77d49fbf73796 consensus state has no leader
I20260927 23:29:00.385118 4622 catalog_manager.cc:5587] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571) does not have the latest schema for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]). Expected version 1 got 0
W20260927 23:29:00.387599 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:29:00.387850 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
I20260927 23:29:00.397923 5221 tablet_bootstrap.cc:492] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:00.398335 5221 ts_tablet_manager.cc:1397] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.068s user 0.008s sys 0.000s
I20260927 23:29:00.400626 5221 raft_consensus.cc:348] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.401253 5221 raft_consensus.cc:374] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.401490 5221 raft_consensus.cc:729] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.402136 5221 consensus_queue.cc:260] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.356114 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:00.403478 5437 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:00.405673 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet e56378a5d42e47d395c881a0898da495 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:00.409734 5229 ts_tablet_manager.cc:1428] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.102s user 0.004s sys 0.003s
I20260927 23:29:00.410624 5229 tablet_bootstrap.cc:492] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:00.412786 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet ffd4fd7dc4b14265b0eafe894b748131 (table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]) successfully deleted
I20260927 23:29:00.418285 5221 ts_tablet_manager.cc:1428] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.020s user 0.006s sys 0.001s
I20260927 23:29:00.424942 5221 tablet_bootstrap.cc:492] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:00.426648 5449 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:00.427068 5449 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.429083 5449 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
W20260927 23:29:00.430681 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:00.431655 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "327adcc2a27f42acb3a77d49fbf73796" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:00.432999 4769 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.434191 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "327adcc2a27f42acb3a77d49fbf73796" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:29:00.435415 4769 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.435799 4769 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:00.436605 5449 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:00.437445 5448 raft_consensus.cc:482] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:00.437902 5448 raft_consensus.cc:504] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.440097 5221 tablet_bootstrap.cc:654] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.441022 5448 leader_election.cc:288] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:00.445776 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet bdecd389ff0341f4ab867eb160786ea9 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:00.446694 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64cfbad1561a41048009b5fc819cc5de" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:00.451889 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
W20260927 23:29:00.455469 4907 leader_election.cc:341] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de
I20260927 23:29:00.458958 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet e56378a5d42e47d395c881a0898da495
I20260927 23:29:00.460845 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet e56378a5d42e47d395c881a0898da495 with delete_type TABLET_DATA_DELETED (replaced by tablet b4ec12a496904c3599bdc001476ebee9) from {username='slave'} at 127.0.0.1:53202
I20260927 23:29:00.461478 5224 tablet_replica.cc:330] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:00.462106 5224 raft_consensus.cc:2229] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:00.462491 5224 raft_consensus.cc:2258] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:00.464035 5224 ts_tablet_manager.cc:1905] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:00.465361 5219 tablet_bootstrap.cc:492] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:00.468593 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet e56378a5d42e47d395c881a0898da495 with delete_type TABLET_DATA_DELETED (replaced by tablet b4ec12a496904c3599bdc001476ebee9) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:00.470995 5219 tablet_bootstrap.cc:654] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:00.471369 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet e56378a5d42e47d395c881a0898da495 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e56378a5d42e47d395c881a0898da495 already in progress: creating tablet
I20260927 23:29:00.475696 4627 catalog_manager.cc:6565] ReplaceTablet: tablet e56378a5d42e47d395c881a0898da495 deleted and replaced by tablet b4ec12a496904c3599bdc001476ebee9
W20260927 23:29:00.477469 4910 leader_election.cc:341] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.477892 4910 leader_election.cc:302] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:00.480818 5448 raft_consensus.cc:2726] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:00.484413 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 128678us
I20260927 23:29:00.484634 4953 tablet_service.cc:1467] Processing CreateTablet for tablet b4ec12a496904c3599bdc001476ebee9 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:00.485606 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4ec12a496904c3599bdc001476ebee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:00.486743 5054 leader_election.cc:302] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:00.487389 5437 raft_consensus.cc:2781] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:00.487785 5437 raft_consensus.cc:686] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:00.488533 5437 consensus_queue.cc:237] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.490417 5221 tablet_bootstrap.cc:492] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:00.490837 5221 ts_tablet_manager.cc:1397] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.066s user 0.012s sys 0.002s
W20260927 23:29:00.494541 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
I20260927 23:29:00.492991 5221 raft_consensus.cc:348] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.496680 5221 raft_consensus.cc:374] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.497239 5221 raft_consensus.cc:729] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.497824 4812 tablet_service.cc:1467] Processing CreateTablet for tablet b4ec12a496904c3599bdc001476ebee9 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:00.498833 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4ec12a496904c3599bdc001476ebee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:00.498525 5221 consensus_queue.cc:260] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.504252 5219 tablet_bootstrap.cc:492] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:00.504676 5219 ts_tablet_manager.cc:1397] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.039s user 0.009s sys 0.001s
I20260927 23:29:00.507440 5096 tablet_service.cc:1514] Processing DeleteTablet for tablet e56378a5d42e47d395c881a0898da495 with delete_type TABLET_DATA_DELETED (replaced by tablet b4ec12a496904c3599bdc001476ebee9) from {username='slave'} at 127.0.0.1:58244
I20260927 23:29:00.507611 5219 raft_consensus.cc:348] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.508215 5219 raft_consensus.cc:374] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.508482 5219 raft_consensus.cc:729] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
W20260927 23:29:00.509596 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
I20260927 23:29:00.512720 5224 ts_tablet_manager.cc:1918] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:00.513068 5224 log.cc:1196] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/e56378a5d42e47d395c881a0898da495
I20260927 23:29:00.513629 5224 ts_tablet_manager.cc:1939] T e56378a5d42e47d395c881a0898da495 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:00.514925 5221 ts_tablet_manager.cc:1428] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.024s user 0.005s sys 0.000s
I20260927 23:29:00.515637 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet e56378a5d42e47d395c881a0898da495 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:29:00.520591 5219 consensus_queue.cc:260] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:00.521544 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:00.523159 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet e56378a5d42e47d395c881a0898da495 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet e56378a5d42e47d395c881a0898da495 already in progress: creating tablet
W20260927 23:29:00.526572 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
W20260927 23:29:00.526801 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:29:00.526978 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
W20260927 23:29:00.527137 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
W20260927 23:29:00.527292 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
I20260927 23:29:00.528614 5219 ts_tablet_manager.cc:1428] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.024s user 0.003s sys 0.002s
I20260927 23:29:00.535905 5221 tablet_bootstrap.cc:492] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:00.546725 5221 tablet_bootstrap.cc:654] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.549888 5449 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:00.550357 5449 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.554060 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309465f419c24549bf5416a4503b9299" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:29:00.555358 4769 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.552091 5449 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:00.560372 5219 tablet_bootstrap.cc:492] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:00.560710 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309465f419c24549bf5416a4503b9299" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:00.561396 5221 tablet_bootstrap.cc:492] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
W20260927 23:29:00.561957 4769 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.562434 5221 ts_tablet_manager.cc:1397] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.027s user 0.011s sys 0.001s
I20260927 23:29:00.562351 4769 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:00.563359 5449 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:00.566051 5219 tablet_bootstrap.cc:654] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.566669 5096 tablet_service.cc:1467] Processing CreateTablet for tablet b4ec12a496904c3599bdc001476ebee9 (DEFAULT_TABLE table=default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:00.567709 5096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4ec12a496904c3599bdc001476ebee9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:00.569548 5229 tablet_bootstrap.cc:654] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.567615 5221 raft_consensus.cc:348] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.570850 5221 raft_consensus.cc:374] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.571240 5221 raft_consensus.cc:729] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.571978 5221 consensus_queue.cc:260] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:00.573758 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:29:00.590977 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
I20260927 23:29:00.594468 5219 tablet_bootstrap.cc:492] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:00.594895 5219 ts_tablet_manager.cc:1397] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.035s user 0.007s sys 0.003s
I20260927 23:29:00.595796 5229 tablet_bootstrap.cc:492] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:00.596166 5229 ts_tablet_manager.cc:1397] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.186s user 0.006s sys 0.008s
I20260927 23:29:00.596956 5221 ts_tablet_manager.cc:1428] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.034s user 0.005s sys 0.000s
I20260927 23:29:00.596994 5219 raft_consensus.cc:348] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.597609 5219 raft_consensus.cc:374] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.597855 5219 raft_consensus.cc:729] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.598383 5219 consensus_queue.cc:260] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:00.602085 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
I20260927 23:29:00.598152 5229 raft_consensus.cc:348] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.602823 5229 raft_consensus.cc:374] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.605731 5219 ts_tablet_manager.cc:1428] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260927 23:29:00.613016 5221 tablet_bootstrap.cc:492] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:00.603080 5229 raft_consensus.cc:729] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.617789 5229 consensus_queue.cc:260] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.630358 5449 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:00.630818 5449 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.632690 5449 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:00.633646 5229 ts_tablet_manager.cc:1428] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.037s user 0.004s sys 0.001s
I20260927 23:29:00.634486 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9873b2e70c2f49b9be88b3cf8cc6c9b1" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:00.636817 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9873b2e70c2f49b9be88b3cf8cc6c9b1" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:00.638098 4769 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.638688 5229 tablet_bootstrap.cc:492] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
W20260927 23:29:00.640256 4769 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.640655 4769 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
W20260927 23:29:00.642340 4685 hms_client.cc:305] Time spent drop HMS table: real 6.180s user 0.000s sys 0.000s
I20260927 23:29:00.642791 4628 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]
I20260927 23:29:00.643412 5449 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:00.649358 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
W20260927 23:29:00.654546 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
I20260927 23:29:00.659978 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 260ab7817da1431ebd80ee7dd00c40b4 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:00.662519 5221 tablet_bootstrap.cc:654] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.685055 5229 tablet_bootstrap.cc:654] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.686893 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 48dcf44bb7b744b9a98626dbfb6f4e6c from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:00.688884 5448 raft_consensus.cc:482] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:00.689410 5448 raft_consensus.cc:504] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.691272 5448 leader_election.cc:288] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:00.693017 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dffe7e40ecb40c68a06f0cf43861777" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:00.693588 4831 raft_consensus.cc:2454] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:00.694509 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dffe7e40ecb40c68a06f0cf43861777" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:00.695899 4907 leader_election.cc:302] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
W20260927 23:29:00.696753 4910 leader_election.cc:341] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.697474 5448 raft_consensus.cc:2781] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:00.697772 5448 raft_consensus.cc:482] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:00.697892 5221 tablet_bootstrap.cc:492] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:00.698072 5448 raft_consensus.cc:3037] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:00.698238 5221 ts_tablet_manager.cc:1397] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.085s user 0.009s sys 0.003s
I20260927 23:29:00.700379 5221 raft_consensus.cc:348] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.701225 5221 raft_consensus.cc:374] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.701469 5221 raft_consensus.cc:729] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.702333 5221 consensus_queue.cc:260] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.706938 5448 raft_consensus.cc:504] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.710392 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dffe7e40ecb40c68a06f0cf43861777" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:00.710984 4831 raft_consensus.cc:3037] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:00.719662 4831 raft_consensus.cc:2454] T 9dffe7e40ecb40c68a06f0cf43861777 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:00.721508 5221 ts_tablet_manager.cc:1428] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.023s user 0.006s sys 0.005s
I20260927 23:29:00.725006 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9dffe7e40ecb40c68a06f0cf43861777" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
W20260927 23:29:00.726408 4910 leader_election.cc:341] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.726943 5229 tablet_bootstrap.cc:492] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:00.727309 5229 ts_tablet_manager.cc:1397] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.089s user 0.003s sys 0.010s
I20260927 23:29:00.727663 5448 leader_election.cc:288] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:00.729367 5229 raft_consensus.cc:348] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.729972 5229 raft_consensus.cc:374] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.730216 5229 raft_consensus.cc:729] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.730448 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet e56378a5d42e47d395c881a0898da495 with delete_type TABLET_DATA_DELETED (replaced by tablet b4ec12a496904c3599bdc001476ebee9) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:00.730530 4907 leader_election.cc:302] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters: f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:00.730840 5229 consensus_queue.cc:260] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.731343 5223 tablet_replica.cc:330] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:00.732241 5223 raft_consensus.cc:2229] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:00.732635 5223 raft_consensus.cc:2258] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:00.733341 5448 raft_consensus.cc:2781] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
W20260927 23:29:00.739030 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
I20260927 23:29:00.742592 5229 ts_tablet_manager.cc:1428] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.015s user 0.004s sys 0.002s
W20260927 23:29:00.748509 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
I20260927 23:29:00.751088 5221 tablet_bootstrap.cc:492] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:00.756610 5229 tablet_bootstrap.cc:492] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:00.765581 5223 ts_tablet_manager.cc:1905] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:00.766040 5209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 23:29:00.769054 5229 tablet_bootstrap.cc:654] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.769824 5209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 23:29:00.770403 5209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 23:29:00.775208 5221 tablet_bootstrap.cc:654] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.781046 5448 raft_consensus.cc:686] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
I20260927 23:29:00.782764 5448 consensus_queue.cc:237] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.790699 5479 raft_consensus.cc:482] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:00.791162 5479 raft_consensus.cc:504] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.793817 5229 tablet_bootstrap.cc:492] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:00.794203 5229 ts_tablet_manager.cc:1397] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.038s user 0.006s sys 0.003s
I20260927 23:29:00.794199 5479 leader_election.cc:288] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:00.794819 5221 tablet_bootstrap.cc:492] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:00.794845 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdecd389ff0341f4ab867eb160786ea9" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:00.795176 5221 ts_tablet_manager.cc:1397] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.048s user 0.006s sys 0.006s
I20260927 23:29:00.793816 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdecd389ff0341f4ab867eb160786ea9" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:00.796690 4831 raft_consensus.cc:2454] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:00.796217 5229 raft_consensus.cc:348] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.800534 4907 leader_election.cc:302] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:00.800957 5229 raft_consensus.cc:374] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260927 23:29:00.801106 4910 leader_election.cc:341] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.801189 5229 raft_consensus.cc:729] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.801409 5479 raft_consensus.cc:2781] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:00.801807 5479 raft_consensus.cc:482] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:00.801852 5229 consensus_queue.cc:260] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.802088 5479 raft_consensus.cc:3037] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:00.812038 5221 raft_consensus.cc:348] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.813926 5221 raft_consensus.cc:374] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.814175 5221 raft_consensus.cc:729] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.813218 5479 raft_consensus.cc:504] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:00.815258 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
I20260927 23:29:00.816624 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdecd389ff0341f4ab867eb160786ea9" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:00.817312 4831 raft_consensus.cc:3037] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:00.823591 4831 raft_consensus.cc:2454] T bdecd389ff0341f4ab867eb160786ea9 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:00.824599 4907 leader_election.cc:302] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:00.825521 5478 raft_consensus.cc:2781] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:00.828125 5229 ts_tablet_manager.cc:1428] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.034s user 0.006s sys 0.000s
I20260927 23:29:00.832218 5223 ts_tablet_manager.cc:1918] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:00.832654 5223 log.cc:1196] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/e56378a5d42e47d395c881a0898da495
I20260927 23:29:00.833346 5223 ts_tablet_manager.cc:1939] T e56378a5d42e47d395c881a0898da495 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:00.835688 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet e56378a5d42e47d395c881a0898da495 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
W20260927 23:29:00.840408 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
I20260927 23:29:00.854986 5478 raft_consensus.cc:686] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
W20260927 23:29:00.855993 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
I20260927 23:29:00.855810 5478 consensus_queue.cc:237] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.861511 5229 tablet_bootstrap.cc:492] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:00.866813 5229 tablet_bootstrap.cc:654] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.871263 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 48dcf44bb7b744b9a98626dbfb6f4e6c from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:00.874490 5479 leader_election.cc:288] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:00.871949 5221 consensus_queue.cc:260] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.875566 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdecd389ff0341f4ab867eb160786ea9" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:00.876773 5221 ts_tablet_manager.cc:1428] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.081s user 0.006s sys 0.000s
W20260927 23:29:00.876816 4910 leader_election.cc:341] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:00.885020 5229 tablet_bootstrap.cc:492] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:00.885069 5221 tablet_bootstrap.cc:492] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:00.885514 5229 ts_tablet_manager.cc:1397] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.024s user 0.004s sys 0.011s
I20260927 23:29:00.897888 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet bdecd389ff0341f4ab867eb160786ea9 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:00.902079 5221 tablet_bootstrap.cc:654] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:00.905027 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
I20260927 23:29:00.888413 5229 raft_consensus.cc:348] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.907666 5229 raft_consensus.cc:374] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.907902 5229 raft_consensus.cc:729] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.908551 5229 consensus_queue.cc:260] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.913419 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet ffd4fd7dc4b14265b0eafe894b748131 with delete_type TABLET_DATA_DELETED (replaced by tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:00.914073 5223 tablet_replica.cc:330] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:00.914745 5223 raft_consensus.cc:2229] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:00.915170 5223 raft_consensus.cc:2258] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:00.917438 5223 ts_tablet_manager.cc:1905] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:29:00.924867 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
I20260927 23:29:00.918493 5229 ts_tablet_manager.cc:1428] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.032s user 0.005s sys 0.000s
I20260927 23:29:00.931559 5229 tablet_bootstrap.cc:492] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
W20260927 23:29:00.938388 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
W20260927 23:29:00.953241 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
I20260927 23:29:00.966827 5223 ts_tablet_manager.cc:1918] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:00.968335 5229 tablet_bootstrap.cc:654] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:00.968575 5223 log.cc:1196] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:29:00.969198 5223 ts_tablet_manager.cc:1939] T ffd4fd7dc4b14265b0eafe894b748131 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:00.971657 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet ffd4fd7dc4b14265b0eafe894b748131 (table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]) successfully deleted
I20260927 23:29:00.979604 5221 tablet_bootstrap.cc:492] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:00.980866 5221 ts_tablet_manager.cc:1397] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.103s user 0.010s sys 0.001s
I20260927 23:29:00.983165 5221 raft_consensus.cc:348] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.983925 5221 raft_consensus.cc:374] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:00.984229 5221 raft_consensus.cc:729] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:00.984963 5221 consensus_queue.cc:260] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:00.989732 5494 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:29:00.990403 5494 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
W20260927 23:29:00.991541 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet df6ea36da9f6455bb399c2229dcb3790
I20260927 23:29:00.991946 5221 ts_tablet_manager.cc:1428] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260927 23:29:00.992997 5221 tablet_bootstrap.cc:492] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:00.994035 5229 tablet_bootstrap.cc:492] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:00.994526 5229 ts_tablet_manager.cc:1397] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.063s user 0.008s sys 0.000s
I20260927 23:29:00.998194 5494 leader_election.cc:288] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
W20260927 23:29:00.998656 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet df6ea36da9f6455bb399c2229dcb3790
I20260927 23:29:01.000389 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 102 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:29:01.003270 5229 raft_consensus.cc:348] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.003990 5229 raft_consensus.cc:374] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:01.004204 5229 raft_consensus.cc:729] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:01.004789 5229 consensus_queue.cc:260] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.006682 5229 ts_tablet_manager.cc:1428] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.012s user 0.004s sys 0.000s
I20260927 23:29:01.006595 4624 catalog_manager.cc:2148] created HMS catalog entry for table [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]
I20260927 23:29:01.010048 5229 tablet_bootstrap.cc:492] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:01.010274 5485 raft_consensus.cc:482] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.010846 5485 raft_consensus.cc:504] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.012835 5485 leader_election.cc:288] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
W20260927 23:29:01.016402 4907 leader_election.cc:341] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b
I20260927 23:29:01.017261 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6846550519ce44fdafbc895ddca98d8b" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:01.020018 5229 tablet_bootstrap.cc:654] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:01.020673 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
W20260927 23:29:01.021912 4910 leader_election.cc:341] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:01.022362 4910 leader_election.cc:302] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:01.023531 5485 raft_consensus.cc:2726] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:01.032032 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:29:01.032264 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
I20260927 23:29:01.035533 5229 tablet_bootstrap.cc:492] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:01.035909 5229 ts_tablet_manager.cc:1397] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.026s user 0.005s sys 0.004s
I20260927 23:29:01.038508 5229 raft_consensus.cc:348] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.039129 5229 raft_consensus.cc:374] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:01.039371 5229 raft_consensus.cc:729] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:01.040035 5229 consensus_queue.cc:260] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.046300 5229 ts_tablet_manager.cc:1428] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260927 23:29:01.050586 5221 tablet_bootstrap.cc:654] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:01.050726 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
W20260927 23:29:01.050933 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
I20260927 23:29:01.057771 5229 tablet_bootstrap.cc:492] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
W20260927 23:29:01.061404 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
I20260927 23:29:01.065464 5229 tablet_bootstrap.cc:654] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:01.076635 5229 tablet_bootstrap.cc:492] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:01.077031 5229 ts_tablet_manager.cc:1397] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.004s
I20260927 23:29:01.079742 5221 tablet_bootstrap.cc:492] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:01.079494 5229 raft_consensus.cc:348] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.080116 5229 raft_consensus.cc:374] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:01.080268 5221 ts_tablet_manager.cc:1397] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.087s user 0.006s sys 0.003s
I20260927 23:29:01.080386 5229 raft_consensus.cc:729] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:01.081065 5229 consensus_queue.cc:260] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.084056 5229 ts_tablet_manager.cc:1428] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260927 23:29:01.084872 5229 tablet_bootstrap.cc:492] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:01.084450 5221 raft_consensus.cc:348] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.085999 5221 raft_consensus.cc:374] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:01.086549 5221 raft_consensus.cc:729] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:01.086652 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 327adcc2a27f42acb3a77d49fbf73796 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:01.087694 5221 consensus_queue.cc:260] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.089615 5229 tablet_bootstrap.cc:654] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:01.092692 5435 raft_consensus.cc:482] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.093145 5435 raft_consensus.cc:504] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.095093 5435 leader_election.cc:288] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:01.097059 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70cb3313148a47f8b6c692222508eb88" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:01.097705 4970 raft_consensus.cc:2454] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:01.099009 5054 leader_election.cc:302] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:01.100257 5229 tablet_bootstrap.cc:492] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:01.100607 5435 raft_consensus.cc:2781] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:01.100713 5229 ts_tablet_manager.cc:1397] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.005s
I20260927 23:29:01.100955 5435 raft_consensus.cc:482] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:01.101311 5435 raft_consensus.cc:3037] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:29:01.102706 5051 leader_election.cc:341] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88
I20260927 23:29:01.110396 5437 raft_consensus.cc:482] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.110484 5221 ts_tablet_manager.cc:1428] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.030s user 0.000s sys 0.004s
I20260927 23:29:01.110811 5437 raft_consensus.cc:504] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.113651 5437 leader_election.cc:288] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:01.117398 5221 tablet_bootstrap.cc:492] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:01.119977 5485 raft_consensus.cc:482] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.120462 5485 raft_consensus.cc:504] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.122290 5485 leader_election.cc:288] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.125933 5229 raft_consensus.cc:348] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.126596 5229 raft_consensus.cc:374] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:01.126495 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2826716e5b184cc2b60d556ca2b89f4d" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:01.126825 5229 raft_consensus.cc:729] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:01.127094 4970 raft_consensus.cc:2454] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:01.127375 5229 consensus_queue.cc:260] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.128264 5054 leader_election.cc:302] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:01.128895 5221 tablet_bootstrap.cc:654] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:01.129405 5436 raft_consensus.cc:2781] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:01.129703 5436 raft_consensus.cc:482] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:01.130036 5436 raft_consensus.cc:3037] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.137140 4833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d90eb9269884933a9af575b5af8a47e" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:01.137753 4833 raft_consensus.cc:2454] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
W20260927 23:29:01.138103 5051 leader_election.cc:341] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d
I20260927 23:29:01.143808 5221 tablet_bootstrap.cc:492] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:01.145061 5221 ts_tablet_manager.cc:1397] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.031s user 0.006s sys 0.005s
I20260927 23:29:01.146586 4907 leader_election.cc:302] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:01.147377 5485 raft_consensus.cc:2781] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:01.147684 5485 raft_consensus.cc:482] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:01.147982 5485 raft_consensus.cc:3037] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.148288 5229 ts_tablet_manager.cc:1428] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.047s user 0.003s sys 0.002s
I20260927 23:29:01.149181 5229 tablet_bootstrap.cc:492] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:01.150616 5221 raft_consensus.cc:348] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.152508 5221 raft_consensus.cc:374] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:01.153285 5221 raft_consensus.cc:729] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:01.154448 5221 consensus_queue.cc:260] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:01.163045 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
W20260927 23:29:01.163321 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
I20260927 23:29:01.164736 5221 ts_tablet_manager.cc:1428] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.019s user 0.004s sys 0.001s
I20260927 23:29:01.168128 5221 tablet_bootstrap.cc:492] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:01.178637 5221 tablet_bootstrap.cc:654] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:01.198331 4537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 6 similar messages]
I20260927 23:29:01.201064 5485 raft_consensus.cc:504] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.202364 5221 tablet_bootstrap.cc:492] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:01.202752 5221 ts_tablet_manager.cc:1397] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.037s user 0.010s sys 0.000s
I20260927 23:29:01.204505 5479 raft_consensus.cc:482] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.204952 4833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d90eb9269884933a9af575b5af8a47e" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:01.204942 5479 raft_consensus.cc:504] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.205490 4833 raft_consensus.cc:3037] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.205686 5221 raft_consensus.cc:348] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.206274 5221 raft_consensus.cc:374] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:01.206504 5221 raft_consensus.cc:729] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:01.206725 5479 leader_election.cc:288] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.207029 5221 consensus_queue.cc:260] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.208864 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48dcf44bb7b744b9a98626dbfb6f4e6c" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:01.209491 4831 raft_consensus.cc:2454] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:01.210701 4907 leader_election.cc:302] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:01.211781 5479 raft_consensus.cc:2781] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:01.212342 5479 raft_consensus.cc:482] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:01.212576 4833 raft_consensus.cc:2454] T 0d90eb9269884933a9af575b5af8a47e P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:01.212710 5479 raft_consensus.cc:3037] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.214793 5221 ts_tablet_manager.cc:1428] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.012s user 0.005s sys 0.000s
I20260927 23:29:01.216626 4907 leader_election.cc:302] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:01.217362 5221 tablet_bootstrap.cc:492] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:01.217330 5478 raft_consensus.cc:2781] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:01.217800 5478 raft_consensus.cc:686] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
I20260927 23:29:01.218541 5478 consensus_queue.cc:237] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.224670 5221 tablet_bootstrap.cc:654] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:01.233866 5479 raft_consensus.cc:504] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.237558 4833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48dcf44bb7b744b9a98626dbfb6f4e6c" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:01.238101 4833 raft_consensus.cc:3037] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.240183 5221 tablet_bootstrap.cc:492] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:01.241117 5221 ts_tablet_manager.cc:1397] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.026s user 0.007s sys 0.003s
I20260927 23:29:01.244755 4833 raft_consensus.cc:2454] T 48dcf44bb7b744b9a98626dbfb6f4e6c P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:01.246204 5221 raft_consensus.cc:348] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.247699 5221 raft_consensus.cc:374] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:01.247824 4907 leader_election.cc:302] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:01.248682 5478 raft_consensus.cc:2781] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:01.248000 5221 raft_consensus.cc:729] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:01.249706 5478 raft_consensus.cc:686] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
I20260927 23:29:01.250036 5221 consensus_queue.cc:260] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.250586 5478 consensus_queue.cc:237] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.256486 5221 ts_tablet_manager.cc:1428] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.015s user 0.005s sys 0.001s
W20260927 23:29:01.268739 4537 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.62:35155, user_credentials={real_user=slave}} blocked reactor thread for 70541.5us
W20260927 23:29:01.271159 4537 leader_election.cc:334] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:29:01.271607 4537 leader_election.cc:302] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f8460f73b40c41a28ec020fc04203564; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, d9ed71d97661411891f2d2de4cdc2269
I20260927 23:29:01.272276 5494 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:29:01.277468 5518 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.278065 5518 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.280181 4623 catalog_manager.cc:2148] created HMS catalog entry for table [id=aea1156e91a84d018912c856e3f89d34]
I20260927 23:29:01.281212 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ec12a496904c3599bdc001476ebee9" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:01.281881 4970 raft_consensus.cc:2454] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 0.
I20260927 23:29:01.282948 4769 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:01.300997 5485 leader_election.cc:288] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.305305 5479 leader_election.cc:288] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.345374 5509 raft_consensus.cc:482] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.345695 5518 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.345853 5509 raft_consensus.cc:504] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.346622 5526 raft_consensus.cc:2781] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:01.346956 5526 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:01.347265 5526 raft_consensus.cc:3037] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.347736 5509 leader_election.cc:288] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:01.350550 5436 raft_consensus.cc:504] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.352489 5436 leader_election.cc:288] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:01.341374 4769 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 58466.4us
I20260927 23:29:01.361790 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77734e994d5d40dfb3efca344d396956" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:01.362459 4970 raft_consensus.cc:2454] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:01.364395 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2826716e5b184cc2b60d556ca2b89f4d" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:01.364935 4970 raft_consensus.cc:3037] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.371523 4970 raft_consensus.cc:2454] T 2826716e5b184cc2b60d556ca2b89f4d P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:01.373258 4624 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35136 (request call id 10) took 9318 ms. Trace:
I20260927 23:29:01.373603 4624 rpcz_store.cc:276] 0927 23:28:52.055108 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:52.055812 (+ 704us) service_pool.cc:227] Handling call
0927 23:28:54.263640 (+2207828us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:54.266035 (+ 2395us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:01.006554 (+6740519us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:01.372531 (+365977us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:29:01.372893 (+ 362us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:29:01.373180 (+ 287us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:01.009680 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:01.012137 (+ 2457us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:01.013736 (+ 1599us) write_op.cc:435] Acquired schema lock
0927 23:29:01.013757 (+ 21us) tablet.cc:660] Decoding operations
0927 23:29:01.014119 (+ 362us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:01.018638 (+ 4519us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:01.018673 (+ 35us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:29:01.022734 (+ 4061us) tablet.cc:699] Row locks acquired
0927 23:29:01.022844 (+ 110us) write_op.cc:260] PREPARE: finished
0927 23:29:01.022985 (+ 141us) write_op.cc:270] Start()
0927 23:29:01.023082 (+ 97us) write_op.cc:276] Timestamp: P: 1790551741022931 usec, L: 0
0927 23:29:01.023097 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:29:01.024714 (+ 1617us) log.cc:842] Serialized 432 byte log entry
0927 23:29:01.273467 (+248753us) op_driver.cc:444] REPLICATION: finished
0927 23:29:01.323675 (+ 50208us) write_op.cc:301] APPLY: starting
0927 23:29:01.323773 (+ 98us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:01.323958 (+ 185us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:01.324112 (+ 154us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:01.324934 (+ 822us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:01.325182 (+ 248us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:01.325260 (+ 78us) write_op.cc:312] APPLY: finished
0927 23:29:01.325726 (+ 466us) log.cc:842] Serialized 28 byte log entry
0927 23:29:01.326565 (+ 839us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:01.326702 (+ 137us) write_op.cc:454] Released schema lock
0927 23:29:01.363318 (+ 36616us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":6376171,"Hive Metastore.run_cpu_time_us":625,"Hive Metastore.run_wall_time_us":363161,"child_traces":[["op",{"apply.queue_time_us":49900,"num_ops":2,"prepare.queue_time_us":1507,"prepare.run_cpu_time_us":2587,"prepare.run_wall_time_us":19429,"raft.queue_time_us":2070,"raft.run_cpu_time_us":586,"raft.run_wall_time_us":587,"replication_time_us":250285,"thread_start_us":54305,"threads_started":2}]]}
I20260927 23:29:01.379127 4624 master_service.cc:946] ReplaceTablet: received request to replace tablet b4ec12a496904c3599bdc001476ebee9 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:01.408727 5526 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.413457 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ec12a496904c3599bdc001476ebee9" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:01.414976 4970 raft_consensus.cc:3037] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.424602 4970 raft_consensus.cc:2454] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 1.
I20260927 23:29:01.426519 4769 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:01.427321 5518 raft_consensus.cc:2781] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:01.430215 5479 raft_consensus.cc:482] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.430657 5479 raft_consensus.cc:504] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.362542 5521 raft_consensus.cc:482] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.432448 5479 leader_election.cc:288] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.432547 5521 raft_consensus.cc:504] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.433619 4625 catalog_manager.cc:2148] created HMS catalog entry for table [id=fa9d9f99dc5d4747a22f79e6a1533674]
I20260927 23:29:01.434466 5521 leader_election.cc:288] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:01.379217 5051 leader_election.cc:341] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956
I20260927 23:29:01.364573 5435 raft_consensus.cc:504] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:01.435225 4907 leader_election.cc:341] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac
I20260927 23:29:01.436216 5435 leader_election.cc:288] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:01.436766 5051 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:40571, user_credentials={real_user=slave}} blocked reactor thread for 57590.5us
W20260927 23:29:01.437381 5051 leader_election.cc:341] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d
I20260927 23:29:01.437815 5054 leader_election.cc:302] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
W20260927 23:29:01.438585 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 50591.9us
I20260927 23:29:01.406071 5229 tablet_bootstrap.cc:654] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:01.440167 5054 leader_election.cc:302] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:01.419850 5534 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.423395 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d90eb9269884933a9af575b5af8a47e" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:01.423784 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d90eb9269884933a9af575b5af8a47e" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:01.424005 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48dcf44bb7b744b9a98626dbfb6f4e6c" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:01.443001 5534 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:01.444890 5435 raft_consensus.cc:2781] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:01.444969 5534 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:01.445279 5435 raft_consensus.cc:482] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:01.445541 5534 raft_consensus.cc:2781] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:01.445580 5435 raft_consensus.cc:3037] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.446012 5534 raft_consensus.cc:686] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:01.446712 5534 consensus_queue.cc:237] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.448649 5479 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.448702 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6846550519ce44fdafbc895ddca98d8b" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:01.449410 4974 raft_consensus.cc:2454] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:01.449147 5479 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.451246 5479 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.451790 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70cb3313148a47f8b6c692222508eb88" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:01.451845 5532 consensus_queue.cc:1035] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:29:01.452334 4974 raft_consensus.cc:3037] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.425005 5527 raft_consensus.cc:482] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.454084 5527 raft_consensus.cc:504] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.456141 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "327adcc2a27f42acb3a77d49fbf73796" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:01.456858 4831 raft_consensus.cc:2454] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:01.457598 5054 leader_election.cc:302] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:01.458328 5519 raft_consensus.cc:2781] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:01.458727 5519 raft_consensus.cc:482] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:01.459055 5519 raft_consensus.cc:3037] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.459273 5526 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Requested vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.461115 4907 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:01.462210 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ec12a496904c3599bdc001476ebee9" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:01.463979 4769 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:01.464783 5479 raft_consensus.cc:2781] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:01.465109 5479 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:01.465416 5479 raft_consensus.cc:3037] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.474233 5527 leader_election.cc:288] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:01.458382 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:29:01.476106 4969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64cfbad1561a41048009b5fc819cc5de" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:01.476969 4969 raft_consensus.cc:2454] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:01.479439 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ec12a496904c3599bdc001476ebee9" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
W20260927 23:29:01.480726 4769 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:01.481793 5054 leader_election.cc:302] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:01.482478 5229 tablet_bootstrap.cc:492] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:01.482846 5229 ts_tablet_manager.cc:1397] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.334s user 0.011s sys 0.019s
I20260927 23:29:01.483057 5521 raft_consensus.cc:2781] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:01.483446 5521 raft_consensus.cc:482] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:01.483825 5521 raft_consensus.cc:3037] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.484443 4974 raft_consensus.cc:2454] T 70cb3313148a47f8b6c692222508eb88 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:01.484833 5114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48dcf44bb7b744b9a98626dbfb6f4e6c" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:01.485246 5229 raft_consensus.cc:348] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.485889 5229 raft_consensus.cc:374] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:01.486126 5229 raft_consensus.cc:729] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:01.486646 5229 consensus_queue.cc:260] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.488561 5229 ts_tablet_manager.cc:1428] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260927 23:29:01.491372 4622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:29:01.492512 5229 tablet_bootstrap.cc:492] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:01.492641 4622 catalog_manager.cc:6565] ReplaceTablet: tablet 327adcc2a27f42acb3a77d49fbf73796 deleted and replaced by tablet b373bde60f734d049c4a45255e2b2e2a
W20260927 23:29:01.493702 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:29:01.494251 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
W20260927 23:29:01.494230 5051 leader_election.cc:341] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b
I20260927 23:29:01.494886 5479 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.495539 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 327adcc2a27f42acb3a77d49fbf73796 with delete_type TABLET_DATA_DELETED (replaced by tablet b373bde60f734d049c4a45255e2b2e2a) from {username='slave'} at 127.0.0.1:53202
I20260927 23:29:01.497391 5229 tablet_bootstrap.cc:654] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:01.510293 5519 raft_consensus.cc:504] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.511763 5479 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.512214 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 327adcc2a27f42acb3a77d49fbf73796 with delete_type TABLET_DATA_DELETED (replaced by tablet b373bde60f734d049c4a45255e2b2e2a) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:01.512847 5229 tablet_bootstrap.cc:492] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:01.513213 5229 ts_tablet_manager.cc:1397] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.004s
W20260927 23:29:01.520671 4910 leader_election.cc:341] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:01.526255 5553 tablet_replica.cc:330] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:01.527084 5479 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.527369 5553 raft_consensus.cc:2229] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:01.527966 5553 raft_consensus.cc:2258] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:01.527567 5479 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.528801 5096 tablet_service.cc:1514] Processing DeleteTablet for tablet 327adcc2a27f42acb3a77d49fbf73796 with delete_type TABLET_DATA_DELETED (replaced by tablet b373bde60f734d049c4a45255e2b2e2a) from {username='slave'} at 127.0.0.1:58244
W20260927 23:29:01.530258 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 327adcc2a27f42acb3a77d49fbf73796 consensus state has no leader
I20260927 23:29:01.530503 5553 ts_tablet_manager.cc:1905] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:01.530557 5479 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.536773 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "327adcc2a27f42acb3a77d49fbf73796" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:01.537355 4831 raft_consensus.cc:3037] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.540850 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9873b2e70c2f49b9be88b3cf8cc6c9b1" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:01.541471 4830 raft_consensus.cc:2454] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:01.542480 4907 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:01.543157 5485 raft_consensus.cc:2781] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:01.543004 5479 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.543489 5485 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:01.543797 5485 raft_consensus.cc:3037] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.543668 5479 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.547250 4831 raft_consensus.cc:2454] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:01.548935 5479 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.549062 5518 raft_consensus.cc:686] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: Becoming Leader. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Running, Role: LEADER
I20260927 23:29:01.550092 5518 consensus_queue.cc:237] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.550297 5485 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.553432 5551 tablet_replica.cc:330] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:01.554585 5485 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.555223 4907 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
W20260927 23:29:01.555917 4907 raft_consensus.cc:2670] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260927 23:29:01.559458 5485 raft_consensus.cc:482] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.559870 5485 raft_consensus.cc:504] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.561391 5553 ts_tablet_manager.cc:1918] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:01.561728 5553 log.cc:1196] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/327adcc2a27f42acb3a77d49fbf73796
I20260927 23:29:01.563017 4623 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35136 (request call id 11) took 9478 ms. Trace:
I20260927 23:29:01.565485 5553 ts_tablet_manager.cc:1939] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:01.565739 5485 leader_election.cc:288] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.563344 4623 rpcz_store.cc:276] 0927 23:28:52.102577 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:52.104701 (+ 2124us) service_pool.cc:227] Handling call
0927 23:28:52.108751 (+ 4050us) spinlock_profiling.cc:243] Waited 1.63 ms on lock 0x7b54000118d4. stack: 00007f14799af033 00007f14799aef30 00007f1479530534 00000000004db2b4 00000000004db249 00007f14808bbaf0 00007f148093ef2a 00007f148280ad6c 00007f1482906156 00007f1480443034 00007f1480442fc2 00007f1480442ef1 00007f1480442e6c 00007f1480442122 00007f147e6e594c 00007f147e6e4d86
0927 23:28:54.268706 (+2159955us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:54.270524 (+ 1818us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:01.280123 (+7009599us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:01.562272 (+282149us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:29:01.562654 (+ 382us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:29:01.562947 (+ 293us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:01.282199 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:01.282658 (+ 459us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:01.282685 (+ 27us) write_op.cc:435] Acquired schema lock
0927 23:29:01.282702 (+ 17us) tablet.cc:660] Decoding operations
0927 23:29:01.283024 (+ 322us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:01.283066 (+ 42us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:01.283097 (+ 31us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:29:01.283333 (+ 236us) tablet.cc:699] Row locks acquired
0927 23:29:01.283351 (+ 18us) write_op.cc:260] PREPARE: finished
0927 23:29:01.283487 (+ 136us) write_op.cc:270] Start()
0927 23:29:01.283578 (+ 91us) write_op.cc:276] Timestamp: P: 1790551741283408 usec, L: 0
0927 23:29:01.283603 (+ 25us) op_driver.cc:331] REPLICATION: starting
0927 23:29:01.327619 (+ 44016us) log.cc:842] Serialized 432 byte log entry
0927 23:29:01.329304 (+ 1685us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b54000118d4. stack: 00007f14799af033 00007f14799aef30 00007f1479530534 00000000004db2b4 00000000004db249 00007f14808bb471 00007f1480932198 00007f1480d93ade 00007f1480d932b8 00007f1480d97ba1 00007f1480d97b59 00007f1480d97ae9 00007f1480d97ab1 00007f1480d96dad 0000000000580784 00000000005805b9
0927 23:29:01.557957 (+228653us) op_driver.cc:444] REPLICATION: finished
0927 23:29:01.558413 (+ 456us) write_op.cc:301] APPLY: starting
0927 23:29:01.558478 (+ 65us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:01.558588 (+ 110us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:01.558606 (+ 18us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:01.559052 (+ 446us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:01.559200 (+ 148us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:01.559242 (+ 42us) write_op.cc:312] APPLY: finished
0927 23:29:01.559456 (+ 214us) log.cc:842] Serialized 28 byte log entry
0927 23:29:01.559612 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:01.559671 (+ 59us) write_op.cc:454] Released schema lock
0927 23:29:01.559700 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:01.559869 (+ 169us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:01.559889 (+ 20us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":6734931,"Hive Metastore.run_cpu_time_us":610,"Hive Metastore.run_wall_time_us":273430,"spinlock_wait_cycles":3752832,"child_traces":[["op",{"apply.queue_time_us":259,"apply.run_cpu_time_us":1654,"apply.run_wall_time_us":1654,"num_ops":2,"prepare.queue_time_us":138,"prepare.run_cpu_time_us":10593,"prepare.run_wall_time_us":78411,"raft.queue_time_us":4092,"raft.run_cpu_time_us":8,"raft.run_wall_time_us":8,"replication_time_us":274272,"spinlock_wait_cycles":99834624,"thread_start_us":31008,"threads_started":1}]]}
I20260927 23:29:01.567219 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:29:01.553516 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309465f419c24549bf5416a4503b9299" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:01.569442 4624 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet b4ec12a496904c3599bdc001476ebee9
I20260927 23:29:01.569695 4831 raft_consensus.cc:2454] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:01.570943 4624 catalog_manager.cc:6565] ReplaceTablet: tablet b4ec12a496904c3599bdc001476ebee9 deleted and replaced by tablet 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:01.575177 4625 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35136 (request call id 12) took 9358 ms. Trace:
I20260927 23:29:01.575512 4625 rpcz_store.cc:276] 0927 23:28:52.217207 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:52.217370 (+ 163us) service_pool.cc:227] Handling call
0927 23:28:54.272914 (+2055544us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:28:54.274898 (+ 1984us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:01.433571 (+7158673us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:01.574426 (+140855us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:29:01.574820 (+ 394us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:29:01.575116 (+ 296us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:01.435477 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:01.435938 (+ 461us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:01.435965 (+ 27us) write_op.cc:435] Acquired schema lock
0927 23:29:01.435978 (+ 13us) tablet.cc:660] Decoding operations
0927 23:29:01.436318 (+ 340us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:01.436364 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:01.436390 (+ 26us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:29:01.436631 (+ 241us) tablet.cc:699] Row locks acquired
0927 23:29:01.436645 (+ 14us) write_op.cc:260] PREPARE: finished
0927 23:29:01.436797 (+ 152us) write_op.cc:270] Start()
0927 23:29:01.436900 (+ 103us) write_op.cc:276] Timestamp: P: 1790551741436752 usec, L: 0
0927 23:29:01.436919 (+ 19us) op_driver.cc:331] REPLICATION: starting
0927 23:29:01.437493 (+ 574us) log.cc:842] Serialized 432 byte log entry
0927 23:29:01.568390 (+130897us) op_driver.cc:444] REPLICATION: finished
0927 23:29:01.568924 (+ 534us) write_op.cc:301] APPLY: starting
0927 23:29:01.568975 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:01.569075 (+ 100us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:01.569163 (+ 88us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:01.573022 (+ 3859us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:01.573155 (+ 133us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:01.573198 (+ 43us) write_op.cc:312] APPLY: finished
0927 23:29:01.573393 (+ 195us) log.cc:842] Serialized 28 byte log entry
0927 23:29:01.574116 (+ 723us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:01.574193 (+ 77us) write_op.cc:454] Released schema lock
0927 23:29:01.574226 (+ 33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":7004130,"Hive Metastore.run_cpu_time_us":618,"Hive Metastore.run_wall_time_us":153613,"child_traces":[["op",{"apply.queue_time_us":360,"num_ops":2,"prepare.queue_time_us":130,"prepare.run_cpu_time_us":2329,"prepare.run_wall_time_us":2329,"raft.queue_time_us":1541,"raft.run_cpu_time_us":19,"raft.run_wall_time_us":20,"replication_time_us":129865}]]}
I20260927 23:29:01.554765 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9873b2e70c2f49b9be88b3cf8cc6c9b1" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:01.582834 4830 raft_consensus.cc:3037] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.589560 5551 raft_consensus.cc:2229] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:01.590051 5551 raft_consensus.cc:2258] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:01.591826 5551 ts_tablet_manager.cc:1905] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:01.597141 4625 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_99f8c479cf8848c08b0136e79c68b658"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:29:01.614042 4810 tablet_service.cc:1467] Processing CreateTablet for tablet c739222645654eb39285a2ad1b6c35f5 (DEFAULT_TABLE table=default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:01.615048 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c739222645654eb39285a2ad1b6c35f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:01.617728 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet b4ec12a496904c3599bdc001476ebee9 with delete_type TABLET_DATA_DELETED (replaced by tablet 1766d227273e4c7592c4f0a20f2f7b17) from {username='slave'} at 127.0.0.1:53202
W20260927 23:29:01.620517 4624 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
W20260927 23:29:01.620922 4624 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:01.621132 4624 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
I20260927 23:29:01.635253 4830 raft_consensus.cc:2454] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:01.637210 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet 327adcc2a27f42acb3a77d49fbf73796 with delete_type TABLET_DATA_DELETED (replaced by tablet b373bde60f734d049c4a45255e2b2e2a) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:01.637215 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet b4ec12a496904c3599bdc001476ebee9 with delete_type TABLET_DATA_DELETED (replaced by tablet 1766d227273e4c7592c4f0a20f2f7b17) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:01.638528 5553 tablet_replica.cc:330] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:01.639343 5553 raft_consensus.cc:2229] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:01.639781 5553 raft_consensus.cc:2258] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:01.642032 5553 ts_tablet_manager.cc:1905] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:01.643216 5551 ts_tablet_manager.cc:1918] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:01.653532 5551 log.cc:1196] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/327adcc2a27f42acb3a77d49fbf73796
W20260927 23:29:01.657181 4910 leader_election.cc:341] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:29:01.658128 4910 leader_election.cc:341] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:01.671152 5551 ts_tablet_manager.cc:1939] T 327adcc2a27f42acb3a77d49fbf73796 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:01.678164 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:29:01.681975 5566 tablet_bootstrap.cc:492] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:01.689190 5551 tablet_replica.cc:330] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:01.694902 5551 raft_consensus.cc:2229] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:29:01.706132 5551 pending_rounds.cc:62] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca: Trying to abort 1 pending ops.
I20260927 23:29:01.709280 5551 pending_rounds.cc:68] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099933462945792 op_type: NO_OP noop_request { }
I20260927 23:29:01.714548 5551 raft_consensus.cc:2866] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:29:01.715981 5551 raft_consensus.cc:2258] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:01.720073 5510 raft_consensus.cc:482] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.723335 5566 tablet_bootstrap.cc:654] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:01.720547 5510 raft_consensus.cc:504] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.512168 5519 leader_election.cc:288] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:01.729048 4907 leader_election.cc:341] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790
I20260927 23:29:01.729533 4953 tablet_service.cc:1467] Processing CreateTablet for tablet c739222645654eb39285a2ad1b6c35f5 (DEFAULT_TABLE table=default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:01.730540 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c739222645654eb39285a2ad1b6c35f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:01.730963 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:01.515936 5521 raft_consensus.cc:504] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:01.733875 4910 leader_election.cc:341] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:29:01.494971 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:29:01.729636 4907 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:01.744851 5229 raft_consensus.cc:348] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.745498 5229 raft_consensus.cc:374] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:01.745738 5229 raft_consensus.cc:729] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:01.745704 5485 raft_consensus.cc:2781] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:01.746026 5485 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:01.746289 5485 raft_consensus.cc:3037] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.746287 5229 consensus_queue.cc:260] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.748828 5519 raft_consensus.cc:482] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.749202 5519 raft_consensus.cc:504] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.746747 5436 raft_consensus.cc:482] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.750650 5436 raft_consensus.cc:504] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.747272 5051 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:01.752851 5485 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.753809 5551 ts_tablet_manager.cc:1905] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:01.755280 5485 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:01.756085 4907 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:01.759184 5485 raft_consensus.cc:2781] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
W20260927 23:29:01.759318 5051 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:40571, user_credentials={real_user=slave}} blocked reactor thread for 264338us
I20260927 23:29:01.763070 5485 raft_consensus.cc:686] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
I20260927 23:29:01.764091 5485 consensus_queue.cc:237] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:01.768318 4617 catalog_manager.cc:4808] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 327adcc2a27f42acb3a77d49fbf73796 because the tablet was not found. No further retry: Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:29:01.769992 4626 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]
I20260927 23:29:01.770905 5479 consensus_queue.cc:1035] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:01.773682 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309465f419c24549bf5416a4503b9299" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:01.777988 5551 ts_tablet_manager.cc:1918] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:29:01.788486 5551 log.cc:1196] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/b4ec12a496904c3599bdc001476ebee9
I20260927 23:29:01.789477 5551 ts_tablet_manager.cc:1939] T b4ec12a496904c3599bdc001476ebee9 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:01.774461 4830 raft_consensus.cc:3037] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.794171 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet b4ec12a496904c3599bdc001476ebee9 (table default.table_d49f20a4a3204c20954377a8d7ac4b4b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:29:01.798108 4830 raft_consensus.cc:2454] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:01.809175 5533 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:01.809777 4907 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:01.815184 4623 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_8a2ddb6dcbbc4b8bba970c3bf5c37848"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:29:01.818826 4622 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_829e227cbcac4bc585663258666cd43a"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:29:01.720129 5435 raft_consensus.cc:504] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.845134 5479 raft_consensus.cc:2781] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:01.855979 5566 tablet_bootstrap.cc:492] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:01.856454 5566 ts_tablet_manager.cc:1397] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.175s user 0.013s sys 0.000s
I20260927 23:29:01.859136 5229 ts_tablet_manager.cc:1428] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.346s user 0.006s sys 0.009s
I20260927 23:29:01.859016 5566 raft_consensus.cc:348] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.859834 5566 raft_consensus.cc:374] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:01.861027 5114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5073ab2faa0042569b11193952cce2ac" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:01.862507 5114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "327adcc2a27f42acb3a77d49fbf73796" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:01.863968 5114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "327adcc2a27f42acb3a77d49fbf73796" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:01.865283 5114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9873b2e70c2f49b9be88b3cf8cc6c9b1" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:01.867364 5519 leader_election.cc:288] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:01.868914 5436 leader_election.cc:288] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:01.880486 5051 leader_election.cc:341] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88
I20260927 23:29:01.881824 5051 leader_election.cc:302] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
W20260927 23:29:01.882495 5051 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.1:40571, user_credentials={real_user=slave}} blocked reactor thread for 122576us
W20260927 23:29:01.883625 5051 leader_election.cc:341] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de
I20260927 23:29:01.860102 5566 raft_consensus.cc:729] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:01.888005 5566 consensus_queue.cc:260] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:01.891547 5051 leader_election.cc:341] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65
W20260927 23:29:01.892230 5051 leader_election.cc:341] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d
W20260927 23:29:01.894433 5051 leader_election.cc:341] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069
W20260927 23:29:01.895280 5051 leader_election.cc:341] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b
I20260927 23:29:01.895500 5532 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:01.895932 5532 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.897825 5532 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:01.898272 5532 raft_consensus.cc:2781] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:01.898708 5532 raft_consensus.cc:686] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:01.899379 5566 ts_tablet_manager.cc:1428] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.043s user 0.006s sys 0.000s
I20260927 23:29:01.899495 5532 consensus_queue.cc:237] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.901691 5521 leader_election.cc:288] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:01.902778 5435 leader_election.cc:288] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:01.907212 5510 leader_election.cc:288] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:01.907846 5479 raft_consensus.cc:686] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
I20260927 23:29:01.908656 5479 consensus_queue.cc:237] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:01.913362 5580 tablet_bootstrap.cc:492] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:01.918931 5114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309465f419c24549bf5416a4503b9299" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:01.924506 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:29:01.925279 5051 leader_election.cc:341] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de
W20260927 23:29:01.925861 5051 leader_election.cc:341] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956
W20260927 23:29:01.928372 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 392332us
W20260927 23:29:01.930333 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 327adcc2a27f42acb3a77d49fbf73796 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 327adcc2a27f42acb3a77d49fbf73796 already in progress: creating tablet
I20260927 23:29:01.932561 5114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9873b2e70c2f49b9be88b3cf8cc6c9b1" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:01.932919 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df6ea36da9f6455bb399c2229dcb3790" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:01.933367 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309465f419c24549bf5416a4503b9299" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:01.936091 5229 tablet_bootstrap.cc:492] T 9dffe7e40ecb40c68a06f0cf43861777 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:01.945442 5118 raft_consensus.cc:3037] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:29:01.951653 4907 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:40571, user_credentials={real_user=slave}} blocked reactor thread for 70742.5us
I20260927 23:29:01.955912 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6846550519ce44fdafbc895ddca98d8b" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:01.956188 5580 tablet_bootstrap.cc:654] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:01.956528 4974 raft_consensus.cc:3037] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.958266 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbdfdfc23b0c4e3d9224ac2815c6cc65" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:01.958875 4970 raft_consensus.cc:2454] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:01.960855 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c3c55f368dd4ba991daded55d2e3069" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:01.961390 4970 raft_consensus.cc:2454] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:01.965279 4974 raft_consensus.cc:2454] T 6846550519ce44fdafbc895ddca98d8b P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:01.965493 5553 ts_tablet_manager.cc:1918] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:01.967600 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77734e994d5d40dfb3efca344d396956" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:01.968632 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "64cfbad1561a41048009b5fc819cc5de" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:01.969228 4974 raft_consensus.cc:3037] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.970767 4970 raft_consensus.cc:3037] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:01.978981 4969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4deca666d4e4d25aae3ed6f13e97c4d" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:01.979601 4969 raft_consensus.cc:2454] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:01.981333 4970 raft_consensus.cc:2454] T 77734e994d5d40dfb3efca344d396956 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
W20260927 23:29:01.996135 4910 leader_election.cc:341] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:01.996588 4910 leader_election.cc:302] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:01.997730 5479 raft_consensus.cc:2726] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:01.975517 5553 log.cc:1196] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/b4ec12a496904c3599bdc001476ebee9
W20260927 23:29:02.001475 4910 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:29:02.003705 4687 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1766d227273e4c7592c4f0a20f2f7b17 consensus state has no leader
I20260927 23:29:02.004561 4810 tablet_service.cc:1467] Processing CreateTablet for tablet a338854f83f7461e8a1c790b31ef4a06 (DEFAULT_TABLE table=default.table_77e2daeb57dc4c1d9e46b4e05c32c6f2 [id=aea1156e91a84d018912c856e3f89d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.005648 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a338854f83f7461e8a1c790b31ef4a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.012956 4811 tablet_service.cc:1467] Processing CreateTablet for tablet b373bde60f734d049c4a45255e2b2e2a (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.013921 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b373bde60f734d049c4a45255e2b2e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.012037 4812 tablet_service.cc:1467] Processing CreateTablet for tablet fe9cb9be216d42fa81948f5d9da3d4d8 (DEFAULT_TABLE table=default.table_14122b0b934a4107ab5120525f201e5c [id=fa9d9f99dc5d4747a22f79e6a1533674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.017287 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe9cb9be216d42fa81948f5d9da3d4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.020004 4953 tablet_service.cc:1467] Processing CreateTablet for tablet a338854f83f7461e8a1c790b31ef4a06 (DEFAULT_TABLE table=default.table_77e2daeb57dc4c1d9e46b4e05c32c6f2 [id=aea1156e91a84d018912c856e3f89d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.020581 4952 tablet_service.cc:1467] Processing CreateTablet for tablet fe9cb9be216d42fa81948f5d9da3d4d8 (DEFAULT_TABLE table=default.table_14122b0b934a4107ab5120525f201e5c [id=fa9d9f99dc5d4747a22f79e6a1533674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.020963 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a338854f83f7461e8a1c790b31ef4a06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.023597 4954 tablet_service.cc:1467] Processing CreateTablet for tablet b373bde60f734d049c4a45255e2b2e2a (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.024580 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b373bde60f734d049c4a45255e2b2e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.026065 5566 tablet_bootstrap.cc:492] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:02.029247 5593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.57.62:35155 every 8 connection(s)
I20260927 23:29:02.031550 5566 tablet_bootstrap.cc:654] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.032140 4813 tablet_service.cc:1467] Processing CreateTablet for tablet 1766d227273e4c7592c4f0a20f2f7b17 (DEFAULT_TABLE table=default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.032565 5209 rpc_server.cc:307] RPC server started. Bound to: 127.2.57.62:35155
I20260927 23:29:02.033174 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1766d227273e4c7592c4f0a20f2f7b17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.036808 5209 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/master-0/data/info.pb
W20260927 23:29:02.028290 4624 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
I20260927 23:29:02.042163 4624 master_service.cc:946] ReplaceTablet: received request to replace tablet 48dcf44bb7b744b9a98626dbfb6f4e6c from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:02.044618 2276 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6z6gmC/build/tsan/bin/kudu as pid 5209
I20260927 23:29:02.042899 4687 catalog_manager.cc:2709] Deleting table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82] in response to Hive Metastore notification log event 7
I20260927 23:29:02.047873 5566 tablet_bootstrap.cc:492] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:02.048269 5566 ts_tablet_manager.cc:1397] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.000s
I20260927 23:29:02.050551 5566 raft_consensus.cc:348] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.051216 5566 raft_consensus.cc:374] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.051460 5566 raft_consensus.cc:729] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.052032 5566 consensus_queue.cc:260] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.055315 5566 ts_tablet_manager.cc:1428] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260927 23:29:02.055483 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:29:02.057451 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 289d65ad2eb64788a6fa7dbc1b48122c) from {username='slave'} at 127.0.0.1:53202
I20260927 23:29:02.058265 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1 deleted and replaced by tablet 289d65ad2eb64788a6fa7dbc1b48122c
I20260927 23:29:02.062151 5551 tablet_replica.cc:330] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:02.062917 5551 raft_consensus.cc:2229] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:02.063339 5551 raft_consensus.cc:2258] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:02.065049 5551 ts_tablet_manager.cc:1905] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:02.077669 5598 sys_catalog.cc:263] Verifying existing consensus state
I20260927 23:29:02.084141 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe9cb9be216d42fa81948f5d9da3d4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.084894 4950 tablet_service.cc:1514] Processing DeleteTablet for tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 289d65ad2eb64788a6fa7dbc1b48122c) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:02.085755 4951 tablet_service.cc:1467] Processing CreateTablet for tablet 1766d227273e4c7592c4f0a20f2f7b17 (DEFAULT_TABLE table=default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.086803 4951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1766d227273e4c7592c4f0a20f2f7b17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.095988 5551 ts_tablet_manager.cc:1918] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:02.096402 5551 log.cc:1196] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:29:02.097023 5551 ts_tablet_manager.cc:1939] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:02.099459 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1 (table default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]) successfully deleted
I20260927 23:29:02.107230 5598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: Bootstrap starting.
I20260927 23:29:02.122277 4974 raft_consensus.cc:2454] T 64cfbad1561a41048009b5fc819cc5de P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
W20260927 23:29:02.153158 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
I20260927 23:29:02.167618 5598 log.cc:824] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: Log is configured to *not* fsync() on all Append() calls
I20260927 23:29:02.180321 4880 heartbeater.cc:347] Connected to a master server at 127.2.57.62:35155
I20260927 23:29:02.196044 5598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 23:29:02.197230 5598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: Bootstrap complete.
I20260927 23:29:02.205549 4627 catalog_manager.cc:5495] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d reported cstate change: term changed from 0 to 1, leader changed from <none> to bad92a10de2c4459827cdbb18854d76d (127.2.57.2). New cstate: current_term: 1 leader_uuid: "bad92a10de2c4459827cdbb18854d76d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260927 23:29:02.208549 4687 catalog_manager.cc:5777] Not sending DeleteTablet requests; no consensus state for tablet 289d65ad2eb64788a6fa7dbc1b48122c
I20260927 23:29:02.216640 5479 consensus_queue.cc:1035] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:02.140582 5553 ts_tablet_manager.cc:1939] T b4ec12a496904c3599bdc001476ebee9 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
W20260927 23:29:02.226373 5592 log.cc:924] Time spent T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: Append to log took a long time: real 0.125s user 0.001s sys 0.000s
I20260927 23:29:02.226697 4880 heartbeater.cc:347] Connected to a master server at 127.2.57.62:35155
I20260927 23:29:02.230942 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet b4ec12a496904c3599bdc001476ebee9 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:29:02.236363 5580 tablet_bootstrap.cc:492] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:02.233450 5598 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:02.236917 5580 ts_tablet_manager.cc:1397] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.324s user 0.009s sys 0.005s
I20260927 23:29:02.237092 5598 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9ed71d97661411891f2d2de4cdc2269, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.238214 5598 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:02.241499 5580 raft_consensus.cc:348] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.257189 5518 raft_consensus.cc:482] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:02.257642 5518 raft_consensus.cc:504] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.259400 5518 leader_election.cc:288] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:02.259889 5604 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:02.260800 5604 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:29:02.261477 5598 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: configured and running, proceeding with master startup.
W20260927 23:29:02.263547 4769 leader_election.cc:341] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a338854f83f7461e8a1c790b31ef4a06
I20260927 23:29:02.273928 5580 raft_consensus.cc:374] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.274192 5566 tablet_bootstrap.cc:492] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:02.274569 5580 raft_consensus.cc:729] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.275334 5580 consensus_queue.cc:260] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.251137 5553 tablet_replica.cc:330] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
W20260927 23:29:02.272316 4910 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:02.281654 5553 raft_consensus.cc:2229] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:29:02.282637 5553 pending_rounds.cc:62] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d: Trying to abort 1 pending ops.
I20260927 23:29:02.282955 5553 pending_rounds.cc:68] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099934274920448 op_type: NO_OP noop_request { }
I20260927 23:29:02.283514 5553 raft_consensus.cc:2866] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:29:02.283950 5553 raft_consensus.cc:2258] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:02.286391 5553 ts_tablet_manager.cc:1905] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:02.290516 5566 tablet_bootstrap.cc:654] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.298288 5021 heartbeater.cc:347] Connected to a master server at 127.2.57.62:35155
I20260927 23:29:02.303140 5553 ts_tablet_manager.cc:1918] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:29:02.303768 5553 log.cc:1196] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:29:02.305092 4910 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:02.306255 4687 catalog_manager.cc:2709] Deleting table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92] in response to Hive Metastore notification log event 11
I20260927 23:29:02.313709 5096 tablet_service.cc:1514] Processing DeleteTablet for tablet b4ec12a496904c3599bdc001476ebee9 with delete_type TABLET_DATA_DELETED (replaced by tablet 1766d227273e4c7592c4f0a20f2f7b17) from {username='slave'} at 127.0.0.1:58244
I20260927 23:29:02.273037 5519 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:29:02.316565 4910 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:02.318368 5581 consensus_queue.cc:1035] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:29:02.278421 5510 raft_consensus.cc:482] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:02.313228 5435 raft_consensus.cc:482] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:29:02.325126 4910 leader_election.cc:341] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:02.325585 4910 leader_election.cc:302] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:02.326274 5581 raft_consensus.cc:2726] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:02.304716 5553 ts_tablet_manager.cc:1939] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:02.328389 5097 tablet_service.cc:1467] Processing CreateTablet for tablet a338854f83f7461e8a1c790b31ef4a06 (DEFAULT_TABLE table=default.table_77e2daeb57dc4c1d9e46b4e05c32c6f2 [id=aea1156e91a84d018912c856e3f89d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.329478 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a338854f83f7461e8a1c790b31ef4a06. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:29:02.367635 4624 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
I20260927 23:29:02.372898 5518 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:02.373350 5518 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.375116 5518 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:02.376849 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c739222645654eb39285a2ad1b6c35f5" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:02.377435 4974 raft_consensus.cc:2454] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 0.
W20260927 23:29:02.377745 4910 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:02.378875 4769 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:02.379700 5518 raft_consensus.cc:2781] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:02.379993 5518 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:02.380285 5518 raft_consensus.cc:3037] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:02.381850 5580 ts_tablet_manager.cc:1428] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.145s user 0.002s sys 0.004s
I20260927 23:29:02.384553 5534 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:02.385015 5534 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:29:02.385653 5566 tablet_bootstrap.cc:492] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:02.386068 5566 ts_tablet_manager.cc:1397] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.112s user 0.006s sys 0.004s
I20260927 23:29:02.388696 5094 tablet_service.cc:1467] Processing CreateTablet for tablet c739222645654eb39285a2ad1b6c35f5 (DEFAULT_TABLE table=default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.339679 5435 raft_consensus.cc:504] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.391597 5518 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.339109 5519 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.393756 4624 master_service.cc:946] ReplaceTablet: received request to replace tablet c739222645654eb39285a2ad1b6c35f5 from {username='slave'} at 127.0.0.1:35136
W20260927 23:29:02.400935 4910 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:02.406015 5580 tablet_bootstrap.cc:492] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:02.407815 5566 raft_consensus.cc:348] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.327148 5510 raft_consensus.cc:504] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.408536 5566 raft_consensus.cc:374] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.408793 5566 raft_consensus.cc:729] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.410027 5510 leader_election.cc:288] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:02.414042 5093 tablet_service.cc:1467] Processing CreateTablet for tablet 1766d227273e4c7592c4f0a20f2f7b17 (DEFAULT_TABLE table=default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.415930 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8328db65eb43ebb79cd9d7a1ddb0d8" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:02.416563 4970 raft_consensus.cc:2454] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:02.419107 5092 tablet_service.cc:1514] Processing DeleteTablet for tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1 with delete_type TABLET_DATA_DELETED (replaced by tablet 289d65ad2eb64788a6fa7dbc1b48122c) from {username='slave'} at 127.0.0.1:58244
I20260927 23:29:02.420090 5566 consensus_queue.cc:260] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.433431 5098 tablet_service.cc:1467] Processing CreateTablet for tablet b373bde60f734d049c4a45255e2b2e2a (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.435760 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c739222645654eb39285a2ad1b6c35f5" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:02.436363 4974 raft_consensus.cc:3037] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:02.439422 5519 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:02.440550 5435 leader_election.cc:288] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:02.403429 5095 tablet_service.cc:1467] Processing CreateTablet for tablet fe9cb9be216d42fa81948f5d9da3d4d8 (DEFAULT_TABLE table=default.table_14122b0b934a4107ab5120525f201e5c [id=fa9d9f99dc5d4747a22f79e6a1533674]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.444566 5094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c739222645654eb39285a2ad1b6c35f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.451941 4974 raft_consensus.cc:2454] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 1.
I20260927 23:29:02.454962 4769 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:02.458101 5054 leader_election.cc:302] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:02.459431 5054 leader_election.cc:302] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:02.461567 5518 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Requested vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:02.448026 5534 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
I20260927 23:29:02.464239 5229 tablet_bootstrap.cc:654] T 9dffe7e40ecb40c68a06f0cf43861777 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.465301 5580 tablet_bootstrap.cc:654] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.467762 5436 raft_consensus.cc:2781] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:02.465473 5621 raft_consensus.cc:2781] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:02.468087 5436 raft_consensus.cc:482] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:02.468477 5436 raft_consensus.cc:3037] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:02.472833 5437 raft_consensus.cc:2781] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:02.473196 5437 raft_consensus.cc:482] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:02.473508 5437 raft_consensus.cc:3037] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:29:02.478116 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:29:02.485877 4953 tablet_service.cc:1467] Processing CreateTablet for tablet 289d65ad2eb64788a6fa7dbc1b48122c (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.486841 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289d65ad2eb64788a6fa7dbc1b48122c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.473724 5566 ts_tablet_manager.cc:1428] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.087s user 0.006s sys 0.000s
I20260927 23:29:02.486302 4833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdecd389ff0341f4ab867eb160786ea9" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:02.488842 5566 tablet_bootstrap.cc:492] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
W20260927 23:29:02.492148 4769 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:29:02.493080 4769 leader_election.cc:341] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Not found: Tablet not found: a338854f83f7461e8a1c790b31ef4a06
I20260927 23:29:02.497283 5566 tablet_bootstrap.cc:654] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.481393 5093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1766d227273e4c7592c4f0a20f2f7b17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.516629 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:02.518191 5510 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:02.517400 5095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe9cb9be216d42fa81948f5d9da3d4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.520689 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b373bde60f734d049c4a45255e2b2e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.533499 5580 tablet_bootstrap.cc:492] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:02.533952 5580 ts_tablet_manager.cc:1397] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.128s user 0.009s sys 0.002s
I20260927 23:29:02.535048 5583 consensus_queue.cc:1035] T bdecd389ff0341f4ab867eb160786ea9 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:29:02.536432 5583 consensus_queue.cc:1035] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:02.536926 4624 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet c739222645654eb39285a2ad1b6c35f5
I20260927 23:29:02.539456 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet c739222645654eb39285a2ad1b6c35f5 with delete_type TABLET_DATA_DELETED (replaced by tablet 26c0a9168b8b44fb9283c97a4206b79b) from {username='slave'} at 127.0.0.1:34620
W20260927 23:29:02.540247 5051 leader_election.cc:341] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8
I20260927 23:29:02.540885 4624 catalog_manager.cc:6565] ReplaceTablet: tablet c739222645654eb39285a2ad1b6c35f5 deleted and replaced by tablet 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:29:02.541980 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:29:02.543154 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:29:02.544536 4624 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:02.544782 4624 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
I20260927 23:29:02.497336 4769 leader_election.cc:302] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:02.546396 4624 master_service.cc:946] ReplaceTablet: received request to replace tablet 309465f419c24549bf5416a4503b9299 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:02.546717 5618 raft_consensus.cc:2726] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:02.547674 4769 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.3:45519, user_credentials={real_user=slave}} blocked reactor thread for 54615.4us
I20260927 23:29:02.468263 5621 raft_consensus.cc:686] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: Becoming Leader. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Running, Role: LEADER
I20260927 23:29:02.550783 5621 consensus_queue.cc:237] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.554216 5580 raft_consensus.cc:348] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.555160 5580 raft_consensus.cc:374] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.555519 5580 raft_consensus.cc:729] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.556792 5580 consensus_queue.cc:260] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:02.559283 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 81197.9us
I20260927 23:29:02.557895 5553 tablet_replica.cc:330] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:02.564152 5553 raft_consensus.cc:2229] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:02.564658 5553 raft_consensus.cc:2258] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 23:29:02.566880 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 327adcc2a27f42acb3a77d49fbf73796 consensus state has no leader
I20260927 23:29:02.567044 5553 ts_tablet_manager.cc:1905] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:02.577965 4813 tablet_service.cc:1467] Processing CreateTablet for tablet 289d65ad2eb64788a6fa7dbc1b48122c (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.579090 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289d65ad2eb64788a6fa7dbc1b48122c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.583559 5580 ts_tablet_manager.cc:1428] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.049s user 0.003s sys 0.005s
I20260927 23:29:02.585431 4687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 309465f419c24549bf5416a4503b9299
W20260927 23:29:02.590119 4621 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1c3db2bbcc0e488ea4b6637492ff4221 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:29:02.590989 4630 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35136 (request call id 5) took 10736 ms. Trace:
I20260927 23:29:02.591344 4630 rpcz_store.cc:276] 0927 23:28:51.854623 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:51.854777 (+ 154us) service_pool.cc:227] Handling call
0927 23:28:51.908772 (+ 53995us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:29:02.587456 (+10678684us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:29:02.589896 (+ 2440us) catalog_manager.cc:3616] Apply alter schema
0927 23:29:02.590786 (+ 890us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":344008,"Hive Metastore.run_cpu_time_us":2847,"Hive Metastore.run_wall_time_us":1707216}
I20260927 23:29:02.595798 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:29:02 UTC) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:02.598508 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 309465f419c24549bf5416a4503b9299 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:29:02 UTC) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:02.606912 4628 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35136 (request call id 7) took 10582 ms. Trace:
I20260927 23:29:02.607228 4628 rpcz_store.cc:276] 0927 23:28:52.024766 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:52.025270 (+ 504us) service_pool.cc:227] Handling call
0927 23:28:54.260791 (+2235521us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:29:02.606731 (+8345940us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":200748,"Hive Metastore.run_cpu_time_us":649,"Hive Metastore.run_wall_time_us":6180249}
I20260927 23:29:02.609476 4629 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35136 (request call id 6) took 10720 ms. Trace:
W20260927 23:29:02.609520 4625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_99f8c479cf8848c08b0136e79c68b658 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:29:02.609815 4629 rpcz_store.cc:276] 0927 23:28:51.889264 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:51.889709 (+ 445us) service_pool.cc:227] Handling call
0927 23:28:52.369264 (+479555us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:29:02.607381 (+10238117us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:29:02.608437 (+ 1056us) catalog_manager.cc:3616] Apply alter schema
0927 23:29:02.609334 (+ 897us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":1606334,"Hive Metastore.run_cpu_time_us":1938,"Hive Metastore.run_wall_time_us":485584}
I20260927 23:29:02.612663 4626 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35136 (request call id 13) took 10321 ms. Trace:
I20260927 23:29:02.612999 4626 rpcz_store.cc:276] 0927 23:28:52.290750 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:52.290920 (+ 170us) service_pool.cc:227] Handling call
0927 23:28:54.275746 (+1984826us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:29:02.612513 (+8336767us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":7156786,"Hive Metastore.run_cpu_time_us":324,"Hive Metastore.run_wall_time_us":336641}
W20260927 23:29:02.615000 4623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_8a2ddb6dcbbc4b8bba970c3bf5c37848 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:29:02.617229 4622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_829e227cbcac4bc585663258666cd43a in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:29:02.624812 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
W20260927 23:29:02.626670 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
I20260927 23:29:02.631387 5604 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:02.631899 5604 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:02.636417 4953 tablet_service.cc:1467] Processing CreateTablet for tablet 26c0a9168b8b44fb9283c97a4206b79b (DEFAULT_TABLE table=default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.637862 5604 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:29:02.639125 5580 tablet_bootstrap.cc:492] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:02.639835 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26c0a9168b8b44fb9283c97a4206b79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.560282 5054 leader_election.cc:302] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:02.649430 5534 raft_consensus.cc:2781] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:02.649905 5534 raft_consensus.cc:686] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:02.650655 5534 consensus_queue.cc:237] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:02.652556 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 92405.9us
W20260927 23:29:02.661681 4769 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:29:02.662855 4626 catalog_manager.cc:5587] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717) does not have the latest schema for tablet 1766d227273e4c7592c4f0a20f2f7b17 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]). Expected version 1 got 0
I20260927 23:29:02.663328 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet c739222645654eb39285a2ad1b6c35f5 with delete_type TABLET_DATA_DELETED (replaced by tablet 26c0a9168b8b44fb9283c97a4206b79b) from {username='slave'} at 127.0.0.1:53202
I20260927 23:29:02.666290 5566 tablet_bootstrap.cc:492] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:02.666721 5566 ts_tablet_manager.cc:1397] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.178s user 0.009s sys 0.003s
I20260927 23:29:02.670379 4811 tablet_service.cc:1514] Processing DeleteTablet for tablet 309465f419c24549bf5416a4503b9299 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:29:02 UTC) from {username='slave'} at 127.0.0.1:53202
I20260927 23:29:02.672890 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 26c0a9168b8b44fb9283c97a4206b79b (DEFAULT_TABLE table=default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:02.674024 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26c0a9168b8b44fb9283c97a4206b79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:02.683588 5566 raft_consensus.cc:348] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.689558 5566 raft_consensus.cc:374] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.689862 5566 raft_consensus.cc:729] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.690017 5580 tablet_bootstrap.cc:654] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.690514 5566 consensus_queue.cc:260] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.692770 5566 ts_tablet_manager.cc:1428] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.026s user 0.008s sys 0.000s
I20260927 23:29:02.693614 5566 tablet_bootstrap.cc:492] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:02.598804 5436 raft_consensus.cc:504] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.698434 5436 leader_election.cc:288] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:02.604146 5437 raft_consensus.cc:504] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.701026 5437 leader_election.cc:288] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:02.702476 5553 ts_tablet_manager.cc:1918] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:02.702865 5553 log.cc:1196] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/c739222645654eb39285a2ad1b6c35f5
I20260927 23:29:02.703504 5553 ts_tablet_manager.cc:1939] T c739222645654eb39285a2ad1b6c35f5 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:02.705476 5553 tablet_replica.cc:330] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:02.706245 5553 raft_consensus.cc:2229] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:29:02.711797 5553 pending_rounds.cc:62] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: Trying to abort 1 pending ops.
I20260927 23:29:02.712028 5553 pending_rounds.cc:68] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099934868590592 op_type: NO_OP noop_request { }
I20260927 23:29:02.712369 5553 raft_consensus.cc:2866] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:29:02.712661 5553 raft_consensus.cc:2258] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:02.708798 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet c739222645654eb39285a2ad1b6c35f5 (table default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]) successfully deleted
I20260927 23:29:02.716784 5566 tablet_bootstrap.cc:654] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:02.717144 4617 catalog_manager.cc:4808] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1 because the tablet was not found. No further retry: Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:29:02.720809 4626 catalog_manager.cc:5587] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717) does not have the latest schema for tablet 1766d227273e4c7592c4f0a20f2f7b17 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]). Expected version 1 got 0
I20260927 23:29:02.721873 5054 leader_election.cc:302] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:02.722955 5054 leader_election.cc:302] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:02.723955 5510 raft_consensus.cc:2781] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:02.724326 5510 raft_consensus.cc:482] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:02.724848 5510 raft_consensus.cc:3037] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:02.734498 5566 tablet_bootstrap.cc:492] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:02.734913 5566 ts_tablet_manager.cc:1397] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.041s user 0.010s sys 0.001s
I20260927 23:29:02.738116 5553 ts_tablet_manager.cc:1905] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:29:02.738860 5051 leader_election.cc:341] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65
W20260927 23:29:02.739636 5051 leader_election.cc:341] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069
I20260927 23:29:02.741703 5229 tablet_bootstrap.cc:492] T 9dffe7e40ecb40c68a06f0cf43861777 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:02.741451 5566 raft_consensus.cc:348] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.742070 5229 ts_tablet_manager.cc:1397] T 9dffe7e40ecb40c68a06f0cf43861777 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.882s user 0.013s sys 0.000s
I20260927 23:29:02.742120 5566 raft_consensus.cc:374] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.742377 5566 raft_consensus.cc:729] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.745908 5566 consensus_queue.cc:260] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.748021 5435 raft_consensus.cc:2781] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:02.748463 5435 raft_consensus.cc:686] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:02.749159 5435 consensus_queue.cc:237] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.748014 5566 ts_tablet_manager.cc:1428] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260927 23:29:02.753691 5229 raft_consensus.cc:348] T 9dffe7e40ecb40c68a06f0cf43861777 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.754315 5229 raft_consensus.cc:374] T 9dffe7e40ecb40c68a06f0cf43861777 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.754545 5229 raft_consensus.cc:729] T 9dffe7e40ecb40c68a06f0cf43861777 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.755055 5229 consensus_queue.cc:260] T 9dffe7e40ecb40c68a06f0cf43861777 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.758496 5580 tablet_bootstrap.cc:492] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:02.758911 5580 ts_tablet_manager.cc:1397] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.120s user 0.016s sys 0.012s
I20260927 23:29:02.760011 5585 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:02.760273 5566 tablet_bootstrap.cc:492] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:02.760444 5585 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.762873 5585 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:02.767555 5566 tablet_bootstrap.cc:654] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.770848 5229 ts_tablet_manager.cc:1428] T 9dffe7e40ecb40c68a06f0cf43861777 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.029s user 0.006s sys 0.000s
I20260927 23:29:02.771678 5229 tablet_bootstrap.cc:492] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:02.772504 5118 raft_consensus.cc:3037] T 9dffe7e40ecb40c68a06f0cf43861777 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:02.775393 5580 raft_consensus.cc:348] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.776090 5580 raft_consensus.cc:374] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.776365 5580 raft_consensus.cc:729] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.777303 5580 consensus_queue.cc:260] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.778623 5229 tablet_bootstrap.cc:654] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.779721 5580 ts_tablet_manager.cc:1428] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.021s user 0.006s sys 0.001s
I20260927 23:29:02.781322 5580 tablet_bootstrap.cc:492] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:02.786819 5566 tablet_bootstrap.cc:492] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:02.787243 5566 ts_tablet_manager.cc:1397] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.004s
W20260927 23:29:02.788687 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 65793.2us
I20260927 23:29:02.789428 5054 leader_election.cc:302] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:02.789664 5566 raft_consensus.cc:348] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.790328 5566 raft_consensus.cc:374] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.790573 5566 raft_consensus.cc:729] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.790635 5054 leader_election.cc:302] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:02.791409 5532 raft_consensus.cc:2781] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:02.791133 5566 consensus_queue.cc:260] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.791829 5532 raft_consensus.cc:686] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:02.792387 5437 raft_consensus.cc:2781] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:02.792748 5437 raft_consensus.cc:482] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:02.793150 5437 raft_consensus.cc:3037] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:02.792820 5532 consensus_queue.cc:237] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.795418 5580 tablet_bootstrap.cc:654] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.797935 5566 ts_tablet_manager.cc:1428] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.010s user 0.002s sys 0.005s
I20260927 23:29:02.805619 4629 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "default.Table_77e2daeb57dc4c1d9e46b4e05c32c6f2" } new_table_name: "default.Table_ecd08889b3cb4f09b30c4588cdceda4b" modify_external_catalogs: true
I20260927 23:29:02.811977 5229 tablet_bootstrap.cc:492] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:02.812383 5229 ts_tablet_manager.cc:1397] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.041s user 0.008s sys 0.001s
I20260927 23:29:02.813220 5639 tablet_replica.cc:330] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:02.813704 4626 catalog_manager.cc:5587] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717) does not have the latest schema for tablet b373bde60f734d049c4a45255e2b2e2a (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]). Expected version 1 got 0
I20260927 23:29:02.813928 5580 tablet_bootstrap.cc:492] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:02.814973 5580 ts_tablet_manager.cc:1397] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.012s
W20260927 23:29:02.815651 4626 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet b373bde60f734d049c4a45255e2b2e2a (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet b373bde60f734d049c4a45255e2b2e2a consensus state has no leader
I20260927 23:29:02.815155 5229 raft_consensus.cc:348] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.817281 5580 raft_consensus.cc:348] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.817862 5229 raft_consensus.cc:374] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.818074 5580 raft_consensus.cc:374] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.818127 5229 raft_consensus.cc:729] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.818334 5580 raft_consensus.cc:729] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.819077 5580 consensus_queue.cc:260] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.821127 5229 consensus_queue.cc:260] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.823164 5229 ts_tablet_manager.cc:1428] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260927 23:29:02.823634 5580 ts_tablet_manager.cc:1428] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260927 23:29:02.823945 5229 tablet_bootstrap.cc:492] T 48dcf44bb7b744b9a98626dbfb6f4e6c P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:02.824524 5580 tablet_bootstrap.cc:492] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:02.826361 4630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "default.Table_23bd70063a25487ab35a76f05dd4b81b" } modify_external_catalogs: true
I20260927 23:29:02.826458 4624 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "default.Table_14122b0b934a4107ab5120525f201e5c" } new_table_name: "default.Table_143d2fa1ac844e81b1392bd3cc4d95c7" modify_external_catalogs: true
I20260927 23:29:02.827148 4630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "default.Table_23bd70063a25487ab35a76f05dd4b81b" } modify_external_catalogs: true
I20260927 23:29:02.828135 4628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "default.Table_e141ccac1ae746388643f3ee5f04cdc1" } modify_external_catalogs: true
I20260927 23:29:02.828872 4628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35136:
table { table_name: "default.Table_e141ccac1ae746388643f3ee5f04cdc1" } modify_external_catalogs: true
I20260927 23:29:02.837780 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 1766d227273e4c7592c4f0a20f2f7b17 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:02.840226 5580 tablet_bootstrap.cc:654] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.848029 5229 tablet_bootstrap.cc:654] T 48dcf44bb7b744b9a98626dbfb6f4e6c P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.858956 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:02.860788 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 1766d227273e4c7592c4f0a20f2f7b17 with delete_type TABLET_DATA_DELETED (replaced by tablet a1ae6ef92e96465fa5a414ada1b4ff51) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:02.861457 4627 catalog_manager.cc:6565] ReplaceTablet: tablet 1766d227273e4c7592c4f0a20f2f7b17 deleted and replaced by tablet a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:29:02.863615 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
I20260927 23:29:02.864548 5229 tablet_bootstrap.cc:492] T 48dcf44bb7b744b9a98626dbfb6f4e6c P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:02.864929 5229 ts_tablet_manager.cc:1397] T 48dcf44bb7b744b9a98626dbfb6f4e6c P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.041s user 0.006s sys 0.003s
I20260927 23:29:02.865590 4627 catalog_manager.cc:5587] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717) does not have the latest schema for tablet b373bde60f734d049c4a45255e2b2e2a (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]). Expected version 1 got 0
I20260927 23:29:02.867666 5229 raft_consensus.cc:348] T 48dcf44bb7b744b9a98626dbfb6f4e6c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.868321 5229 raft_consensus.cc:374] T 48dcf44bb7b744b9a98626dbfb6f4e6c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.868556 5229 raft_consensus.cc:729] T 48dcf44bb7b744b9a98626dbfb6f4e6c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.871381 5118 raft_consensus.cc:3037] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:29:02.873140 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
W20260927 23:29:02.873381 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
I20260927 23:29:02.876777 5553 ts_tablet_manager.cc:1918] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:29:02.877215 5553 log.cc:1196] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/309465f419c24549bf5416a4503b9299
I20260927 23:29:02.883460 5566 tablet_bootstrap.cc:492] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:02.885035 5229 consensus_queue.cc:260] T 48dcf44bb7b744b9a98626dbfb6f4e6c P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.886515 5229 ts_tablet_manager.cc:1428] T 48dcf44bb7b744b9a98626dbfb6f4e6c P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.021s user 0.004s sys 0.000s
I20260927 23:29:02.886312 5510 raft_consensus.cc:504] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.887312 5229 tablet_bootstrap.cc:492] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:02.888827 5566 tablet_bootstrap.cc:654] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.889631 4627 catalog_manager.cc:5587] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571) does not have the latest schema for tablet b373bde60f734d049c4a45255e2b2e2a (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]). Expected version 1 got 0
I20260927 23:29:02.893769 5585 consensus_queue.cc:1035] T 9dffe7e40ecb40c68a06f0cf43861777 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:02.895298 5553 ts_tablet_manager.cc:1939] T 309465f419c24549bf5416a4503b9299 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:02.895381 5229 tablet_bootstrap.cc:654] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:02.895814 4910 consensus_peers.cc:487] T 48dcf44bb7b744b9a98626dbfb6f4e6c P bad92a10de2c4459827cdbb18854d76d -> Peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Couldn't send request to peer f5ec98b2972d44838fa273eb7a10d01a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:02.897948 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet 309465f419c24549bf5416a4503b9299 (table default.table_86bcd06c32004328b4e6830ef49b41c5 [id=0aa4192ae5f3479d9df14729b8604f92]) successfully deleted
I20260927 23:29:02.898820 5585 consensus_queue.cc:1035] T 0d90eb9269884933a9af575b5af8a47e P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:02.904978 5229 tablet_bootstrap.cc:492] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:02.905408 5229 ts_tablet_manager.cc:1397] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.003s
I20260927 23:29:02.906898 4626 master_service.cc:946] ReplaceTablet: received request to replace tablet b373bde60f734d049c4a45255e2b2e2a from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:02.907526 5229 raft_consensus.cc:348] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.909759 5639 raft_consensus.cc:2229] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:29:02.910249 5566 tablet_bootstrap.cc:492] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:02.910629 5639 pending_rounds.cc:62] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca: Trying to abort 1 pending ops.
I20260927 23:29:02.910638 5566 ts_tablet_manager.cc:1397] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.027s user 0.009s sys 0.005s
I20260927 23:29:02.911049 5639 pending_rounds.cc:68] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099937882755072 op_type: NO_OP noop_request { }
I20260927 23:29:02.911507 5639 raft_consensus.cc:2866] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:29:02.911921 5639 raft_consensus.cc:2258] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:02.914325 4833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1766d227273e4c7592c4f0a20f2f7b17" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:02.914909 4833 raft_consensus.cc:2454] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:02.908135 5229 raft_consensus.cc:374] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.916163 5229 raft_consensus.cc:729] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.916947 5229 consensus_queue.cc:260] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.918642 5553 tablet_replica.cc:330] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:02.918903 5229 ts_tablet_manager.cc:1428] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260927 23:29:02.919840 5229 tablet_bootstrap.cc:492] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:02.924476 5229 tablet_bootstrap.cc:654] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:02.924235 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 132583us
I20260927 23:29:02.922750 5553 raft_consensus.cc:2229] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:02.930223 4626 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:02.934612 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet b373bde60f734d049c4a45255e2b2e2a with delete_type TABLET_DATA_DELETED (replaced by tablet 1c89dab9709541c1afc6a8cdcd43a118) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:02.935678 4626 catalog_manager.cc:6565] ReplaceTablet: tablet b373bde60f734d049c4a45255e2b2e2a deleted and replaced by tablet 1c89dab9709541c1afc6a8cdcd43a118
I20260927 23:29:02.938002 5553 raft_consensus.cc:2258] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:02.930732 5519 raft_consensus.cc:482] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:02.940522 5519 raft_consensus.cc:504] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.940917 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:29:02.942261 5519 leader_election.cc:288] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:02.931888 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:02.935581 5229 tablet_bootstrap.cc:492] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:02.943730 5229 ts_tablet_manager.cc:1397] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.008s
W20260927 23:29:02.946766 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet b4ec12a496904c3599bdc001476ebee9 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b4ec12a496904c3599bdc001476ebee9 already in progress: creating tablet
I20260927 23:29:02.950659 5229 raft_consensus.cc:348] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.950951 4966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bbdfdfc23b0c4e3d9224ac2815c6cc65" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
W20260927 23:29:02.951326 4910 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:02.951537 5229 raft_consensus.cc:374] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:02.951668 4966 raft_consensus.cc:3037] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:02.951756 5229 raft_consensus.cc:729] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:02.952589 5229 consensus_queue.cc:260] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.954253 5229 ts_tablet_manager.cc:1428] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.010s user 0.000s sys 0.004s
I20260927 23:29:02.955008 5229 tablet_bootstrap.cc:492] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:02.948158 4971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bdecd389ff0341f4ab867eb160786ea9" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:02.961114 4971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c3c55f368dd4ba991daded55d2e3069" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:02.961764 4971 raft_consensus.cc:3037] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:02.964596 4621 catalog_manager.cc:2148] created HMS catalog entry for table [id=09f1820cca0c454e928a02f0677e7c9d]
I20260927 23:29:02.969375 4907 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters: f5ec98b2972d44838fa273eb7a10d01a
W20260927 23:29:02.970218 4907 raft_consensus.cc:2670] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260927 23:29:02.971360 5229 tablet_bootstrap.cc:654] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:02.979847 5639 ts_tablet_manager.cc:1905] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:02.980834 4621 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35136 (request call id 16) took 10489 ms. Trace:
I20260927 23:29:02.981323 4966 raft_consensus.cc:2454] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:02.982460 5621 raft_consensus.cc:482] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:02.982880 5621 raft_consensus.cc:504] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:02.984732 5621 leader_election.cc:288] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:02.985244 5510 leader_election.cc:288] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:02.985986 5117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9cb9be216d42fa81948f5d9da3d4d8" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:02.987990 4769 leader_election.cc:341] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:02.990227 5553 ts_tablet_manager.cc:1905] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:02.981201 4621 rpcz_store.cc:276] 0927 23:28:52.491662 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:28:52.491814 (+ 152us) service_pool.cc:227] Handling call
0927 23:29:02.590660 (+10098846us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:29:02.599527 (+ 8867us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:02.964550 (+365023us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:02.979975 (+ 15425us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:29:02.980434 (+ 459us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:29:02.980749 (+ 315us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:02.966984 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:02.967466 (+ 482us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:02.967499 (+ 33us) write_op.cc:435] Acquired schema lock
0927 23:29:02.967512 (+ 13us) tablet.cc:660] Decoding operations
0927 23:29:02.967863 (+ 351us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:02.967912 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:02.967937 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:29:02.968178 (+ 241us) tablet.cc:699] Row locks acquired
0927 23:29:02.968191 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:29:02.968372 (+ 181us) write_op.cc:270] Start()
0927 23:29:02.968473 (+ 101us) write_op.cc:276] Timestamp: P: 1790551742968266 usec, L: 0
0927 23:29:02.968489 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:29:02.969078 (+ 589us) log.cc:842] Serialized 432 byte log entry
0927 23:29:02.977665 (+ 8587us) op_driver.cc:444] REPLICATION: finished
0927 23:29:02.978419 (+ 754us) write_op.cc:301] APPLY: starting
0927 23:29:02.978483 (+ 64us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:02.978601 (+ 118us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:02.978694 (+ 93us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:02.979113 (+ 419us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:02.979256 (+ 143us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:02.979301 (+ 45us) write_op.cc:312] APPLY: finished
0927 23:29:02.979501 (+ 200us) log.cc:842] Serialized 28 byte log entry
0927 23:29:02.979651 (+ 150us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:02.979729 (+ 78us) write_op.cc:454] Released schema lock
0927 23:29:02.979763 (+ 34us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"Hive Metastore.queue_time_us":5947,"Hive Metastore.run_cpu_time_us":602,"Hive Metastore.run_wall_time_us":357804,"child_traces":[["op",{"apply.queue_time_us":572,"num_ops":2,"prepare.queue_time_us":433,"prepare.run_cpu_time_us":2426,"prepare.run_wall_time_us":2426,"raft.queue_time_us":1108,"raft.run_cpu_time_us":722,"raft.run_wall_time_us":724,"replication_time_us":9070}]]}
I20260927 23:29:02.992416 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet 1766d227273e4c7592c4f0a20f2f7b17 with delete_type TABLET_DATA_DELETED (replaced by tablet a1ae6ef92e96465fa5a414ada1b4ff51) from {username='slave'} at 127.0.0.1:53202
I20260927 23:29:02.993014 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet b373bde60f734d049c4a45255e2b2e2a with delete_type TABLET_DATA_DELETED (replaced by tablet 1c89dab9709541c1afc6a8cdcd43a118) from {username='slave'} at 127.0.0.1:53202
I20260927 23:29:02.996764 4966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9cb9be216d42fa81948f5d9da3d4d8" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:02.997602 4966 raft_consensus.cc:2454] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 0.
I20260927 23:29:02.998857 4769 leader_election.cc:302] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters: f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:03.000017 5621 raft_consensus.cc:2781] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:03.000362 5621 raft_consensus.cc:482] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:03.000654 5621 raft_consensus.cc:3037] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:03.001827 5437 raft_consensus.cc:504] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:03.008826 4621 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
I20260927 23:29:03.008790 5621 raft_consensus.cc:504] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.010775 5621 leader_election.cc:288] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Requested vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:03.012567 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9cb9be216d42fa81948f5d9da3d4d8" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:03.013181 4972 raft_consensus.cc:3037] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:03.020665 4972 raft_consensus.cc:2454] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 1.
I20260927 23:29:03.022324 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d90eb9269884933a9af575b5af8a47e" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:03.022454 4769 leader_election.cc:302] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
W20260927 23:29:03.027153 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:29:03.027554 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
I20260927 23:29:03.027751 4971 raft_consensus.cc:2454] T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:03.033058 5580 tablet_bootstrap.cc:492] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:03.033587 5580 ts_tablet_manager.cc:1397] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.209s user 0.009s sys 0.024s
I20260927 23:29:03.038398 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9cb9be216d42fa81948f5d9da3d4d8" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
W20260927 23:29:03.039820 4769 leader_election.cc:341] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:03.043527 5621 raft_consensus.cc:2781] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:03.045298 5621 raft_consensus.cc:686] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: Becoming Leader. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Running, Role: LEADER
I20260927 23:29:03.046159 5621 consensus_queue.cc:237] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:03.056648 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
W20260927 23:29:03.056932 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:29:03.057250 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
W20260927 23:29:03.057408 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
W20260927 23:29:03.057559 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:29:03.057732 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:29:03.057883 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
W20260927 23:29:03.058045 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
W20260927 23:29:03.058208 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:29:03.058375 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
W20260927 23:29:03.058538 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
W20260927 23:29:03.058712 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
W20260927 23:29:03.058879 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
I20260927 23:29:03.064721 4953 tablet_service.cc:1467] Processing CreateTablet for tablet a1ae6ef92e96465fa5a414ada1b4ff51 (DEFAULT_TABLE table=default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:03.065889 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ae6ef92e96465fa5a414ada1b4ff51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:03.071000 5510 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.071502 5510 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:03.073967 5510 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:03.082743 5165 heartbeater.cc:347] Connected to a master server at 127.2.57.62:35155
I20260927 23:29:03.082861 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet 309465f419c24549bf5416a4503b9299 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:29:02 UTC) from {username='slave'} at 127.0.0.1:58244
I20260927 23:29:03.084893 4966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d90eb9269884933a9af575b5af8a47e" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:03.084784 4560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:29:03.092917 5471 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:03.093420 5093 tablet_service.cc:1514] Processing DeleteTablet for tablet c739222645654eb39285a2ad1b6c35f5 with delete_type TABLET_DATA_DELETED (replaced by tablet 26c0a9168b8b44fb9283c97a4206b79b) from {username='slave'} at 127.0.0.1:58244
I20260927 23:29:03.094259 5604 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:29:03.096958 5051 leader_election.cc:341] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d
W20260927 23:29:03.098608 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:29:03.100633 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:29:03.100967 4812 tablet_service.cc:1467] Processing CreateTablet for tablet a1ae6ef92e96465fa5a414ada1b4ff51 (DEFAULT_TABLE table=default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:03.101899 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ae6ef92e96465fa5a414ada1b4ff51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:03.104755 5437 leader_election.cc:288] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:03.106021 5095 tablet_service.cc:1467] Processing CreateTablet for tablet 289d65ad2eb64788a6fa7dbc1b48122c (DEFAULT_TABLE table=default.table_72baadd8af844c73bdd4465219a8601f [id=409d40c7e476447caaf9bb66f61e4b82]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:03.107108 5095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 289d65ad2eb64788a6fa7dbc1b48122c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:03.112555 5229 tablet_bootstrap.cc:492] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:03.112962 5229 ts_tablet_manager.cc:1397] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.158s user 0.021s sys 0.023s
I20260927 23:29:03.118407 5229 raft_consensus.cc:348] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.119125 5229 raft_consensus.cc:374] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:03.119372 5229 raft_consensus.cc:729] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.121865 5229 consensus_queue.cc:260] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.128386 5229 ts_tablet_manager.cc:1428] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.015s user 0.006s sys 0.001s
I20260927 23:29:03.130132 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 26c0a9168b8b44fb9283c97a4206b79b (DEFAULT_TABLE table=default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:03.131129 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26c0a9168b8b44fb9283c97a4206b79b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:03.137148 5229 tablet_bootstrap.cc:492] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:03.139686 5664 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 23:29:03.140741 4626 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d90eb9269884933a9af575b5af8a47e from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:03.144551 5093 tablet_service.cc:1514] Processing DeleteTablet for tablet 1766d227273e4c7592c4f0a20f2f7b17 with delete_type TABLET_DATA_DELETED (replaced by tablet a1ae6ef92e96465fa5a414ada1b4ff51) from {username='slave'} at 127.0.0.1:58244
I20260927 23:29:03.146314 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet b373bde60f734d049c4a45255e2b2e2a with delete_type TABLET_DATA_DELETED (replaced by tablet 1c89dab9709541c1afc6a8cdcd43a118) from {username='slave'} at 127.0.0.1:58244
I20260927 23:29:03.147926 5098 tablet_service.cc:1467] Processing CreateTablet for tablet a1ae6ef92e96465fa5a414ada1b4ff51 (DEFAULT_TABLE table=default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:03.149089 5229 tablet_bootstrap.cc:654] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:03.150867 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 309465f419c24549bf5416a4503b9299 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 309465f419c24549bf5416a4503b9299 already in progress: creating tablet
I20260927 23:29:03.155601 4966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4deca666d4e4d25aae3ed6f13e97c4d" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:03.157823 4966 raft_consensus.cc:3037] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:03.158386 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ae6ef92e96465fa5a414ada1b4ff51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:03.158243 4627 catalog_manager.cc:5495] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e62b4926a8448b2b5f4923553328bca (127.2.57.1). New cstate: current_term: 1 leader_uuid: "7e62b4926a8448b2b5f4923553328bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260927 23:29:03.165009 4966 raft_consensus.cc:2454] T c4deca666d4e4d25aae3ed6f13e97c4d P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:03.166877 4626 master_service.cc:946] ReplaceTablet: received request to replace tablet a338854f83f7461e8a1c790b31ef4a06 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:03.180830 5229 tablet_bootstrap.cc:492] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:03.181243 5229 ts_tablet_manager.cc:1397] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.001s
I20260927 23:29:03.183660 5229 raft_consensus.cc:348] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.184320 5229 raft_consensus.cc:374] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:03.184556 5229 raft_consensus.cc:729] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.193231 5229 consensus_queue.cc:260] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.195351 5229 ts_tablet_manager.cc:1428] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260927 23:29:03.196148 5229 tablet_bootstrap.cc:492] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:03.196869 5621 raft_consensus.cc:482] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.197369 5621 raft_consensus.cc:504] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.198783 4626 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet a338854f83f7461e8a1c790b31ef4a06
W20260927 23:29:03.199963 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet c739222645654eb39285a2ad1b6c35f5 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet c739222645654eb39285a2ad1b6c35f5 already in progress: creating tablet
I20260927 23:29:03.200210 4626 catalog_manager.cc:6565] ReplaceTablet: tablet a338854f83f7461e8a1c790b31ef4a06 deleted and replaced by tablet 1572bbcfebce4d218b1a0f38ecab0384
I20260927 23:29:03.200752 5229 tablet_bootstrap.cc:654] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:03.200819 5621 leader_election.cc:288] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:03.201972 4621 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_68574d2015dd47bbb92137d783bad572"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:29:03.203666 4769 leader_election.cc:341] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Not found: Tablet not found: 289d65ad2eb64788a6fa7dbc1b48122c
I20260927 23:29:03.162635 5566 raft_consensus.cc:348] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:03.211651 5566 raft_consensus.cc:374] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260927 23:29:03.219889 5051 leader_election.cc:341] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8
I20260927 23:29:03.220474 5229 tablet_bootstrap.cc:492] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:03.221043 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet a338854f83f7461e8a1c790b31ef4a06 with delete_type TABLET_DATA_DELETED (replaced by tablet 1572bbcfebce4d218b1a0f38ecab0384) from {username='slave'} at 127.0.0.1:53202
W20260927 23:29:03.221652 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:29:03.221992 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:03.222689 5519 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:03.226593 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:29:03.226933 5054 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:03.212026 5566 raft_consensus.cc:729] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.228392 5566 consensus_queue.cc:260] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:03.232570 5229 ts_tablet_manager.cc:1397] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.001s
I20260927 23:29:03.186261 5580 raft_consensus.cc:348] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.202188 4952 tablet_service.cc:1514] Processing DeleteTablet for tablet a338854f83f7461e8a1c790b31ef4a06 with delete_type TABLET_DATA_DELETED (replaced by tablet 1572bbcfebce4d218b1a0f38ecab0384) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:03.203727 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289d65ad2eb64788a6fa7dbc1b48122c" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:03.214373 5553 ts_tablet_manager.cc:1918] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:03.244886 5553 log.cc:1196] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:29:03.245461 4769 leader_election.cc:341] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:03.245639 5553 ts_tablet_manager.cc:1939] T 1766d227273e4c7592c4f0a20f2f7b17 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:03.245895 4769 leader_election.cc:302] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:03.247637 5621 raft_consensus.cc:2726] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:03.217831 4968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c8328db65eb43ebb79cd9d7a1ddb0d8" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:03.248323 5580 raft_consensus.cc:374] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:03.248553 4968 raft_consensus.cc:3037] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:03.248677 5580 raft_consensus.cc:729] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.249902 5566 ts_tablet_manager.cc:1428] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.338s user 0.006s sys 0.000s
I20260927 23:29:03.249501 5580 consensus_queue.cc:260] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.253404 5580 ts_tablet_manager.cc:1428] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.220s user 0.008s sys 0.000s
I20260927 23:29:03.254354 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet 1766d227273e4c7592c4f0a20f2f7b17 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:29:03.248220 5553 tablet_replica.cc:330] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:03.264335 5553 raft_consensus.cc:2229] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:03.268788 5553 raft_consensus.cc:2258] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:03.269876 5566 tablet_bootstrap.cc:492] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:03.271445 5553 ts_tablet_manager.cc:1905] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:03.273634 5510 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:03.272569 4968 raft_consensus.cc:2454] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:03.274405 5229 raft_consensus.cc:348] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:03.274741 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 48179.9us
I20260927 23:29:03.275084 5229 raft_consensus.cc:374] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:03.275328 5229 raft_consensus.cc:729] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.275846 5229 consensus_queue.cc:260] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.278236 5229 ts_tablet_manager.cc:1428] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.045s user 0.005s sys 0.000s
I20260927 23:29:03.279062 5229 tablet_bootstrap.cc:492] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:03.283684 5229 tablet_bootstrap.cc:654] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:03.319766 5054 leader_election.cc:302] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:03.325857 5580 tablet_bootstrap.cc:492] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:03.328665 5229 tablet_bootstrap.cc:492] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:03.331303 5510 raft_consensus.cc:2726] T bdecd389ff0341f4ab867eb160786ea9 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:29:03.334143 5532 consensus_queue.cc:1035] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:03.337009 5229 ts_tablet_manager.cc:1397] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.058s user 0.014s sys 0.004s
I20260927 23:29:03.342907 5229 raft_consensus.cc:348] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.343806 5229 raft_consensus.cc:374] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:03.335556 5054 leader_election.cc:302] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:03.355202 5510 raft_consensus.cc:2781] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:03.356158 5510 raft_consensus.cc:686] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:03.357055 5510 consensus_queue.cc:237] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.344162 5229 raft_consensus.cc:729] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.364233 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet a1ae6ef92e96465fa5a414ada1b4ff51 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:03.368353 5054 leader_election.cc:302] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:03.369200 5532 raft_consensus.cc:2781] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:03.369619 5054 leader_election.cc:302] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:03.369624 5532 raft_consensus.cc:686] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:03.371119 5532 consensus_queue.cc:237] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.372134 5580 tablet_bootstrap.cc:654] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:03.376941 5435 raft_consensus.cc:2726] T 0d90eb9269884933a9af575b5af8a47e P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:29:03.363229 5229 consensus_queue.cc:260] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.379221 5553 ts_tablet_manager.cc:1918] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:03.384053 5553 log.cc:1196] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:03.390735 5229 ts_tablet_manager.cc:1428] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.053s user 0.007s sys 0.000s
I20260927 23:29:03.391820 5229 tablet_bootstrap.cc:492] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:03.392602 5553 ts_tablet_manager.cc:1939] T b373bde60f734d049c4a45255e2b2e2a P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:03.393388 4627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:03.394861 4627 catalog_manager.cc:6565] ReplaceTablet: tablet a1ae6ef92e96465fa5a414ada1b4ff51 deleted and replaced by tablet b9f07e8a37404636bda765146b20fc60
I20260927 23:29:03.395217 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ae6ef92e96465fa5a414ada1b4ff51 with delete_type TABLET_DATA_DELETED (replaced by tablet b9f07e8a37404636bda765146b20fc60) from {username='slave'} at 127.0.0.1:53202
I20260927 23:29:03.395892 5553 tablet_replica.cc:330] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:03.397013 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet b373bde60f734d049c4a45255e2b2e2a (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:29:03.401412 5534 consensus_queue.cc:1035] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:03.402678 5519 raft_consensus.cc:482] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:29:03.403136 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:29:03.403586 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:03.404238 5534 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:03.403206 5519 raft_consensus.cc:504] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.405231 5054 leader_election.cc:302] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:03.405995 5534 raft_consensus.cc:2781] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:03.406415 5534 raft_consensus.cc:686] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:03.406536 5519 leader_election.cc:288] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:03.407179 5534 consensus_queue.cc:237] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.402141 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ae6ef92e96465fa5a414ada1b4ff51 with delete_type TABLET_DATA_DELETED (replaced by tablet b9f07e8a37404636bda765146b20fc60) from {username='slave'} at 127.0.0.1:34620
W20260927 23:29:03.410491 5051 leader_election.cc:341] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac
I20260927 23:29:03.411731 5054 leader_election.cc:302] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:03.412526 5534 raft_consensus.cc:2781] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:03.413143 5534 raft_consensus.cc:686] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:03.414062 5534 consensus_queue.cc:237] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.415378 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 260ab7817da1431ebd80ee7dd00c40b4 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:03.424054 5580 tablet_bootstrap.cc:492] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:03.425462 5580 ts_tablet_manager.cc:1397] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.099s user 0.009s sys 0.003s
I20260927 23:29:03.425685 5566 tablet_bootstrap.cc:654] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:03.428462 5580 raft_consensus.cc:348] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.429183 5580 raft_consensus.cc:374] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:03.429464 5580 raft_consensus.cc:729] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.430121 5553 raft_consensus.cc:2229] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:03.430644 5553 raft_consensus.cc:2258] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:03.436050 5229 tablet_bootstrap.cc:654] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:03.437877 5639 ts_tablet_manager.cc:1918] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:29:03.438419 5639 log.cc:1196] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/c739222645654eb39285a2ad1b6c35f5
I20260927 23:29:03.439514 5639 ts_tablet_manager.cc:1939] T c739222645654eb39285a2ad1b6c35f5 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:03.443192 5639 tablet_replica.cc:330] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:03.443048 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet c739222645654eb39285a2ad1b6c35f5 (table default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]) successfully deleted
I20260927 23:29:03.445041 5639 raft_consensus.cc:2229] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus shutting down.
W20260927 23:29:03.447557 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 1766d227273e4c7592c4f0a20f2f7b17 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 1766d227273e4c7592c4f0a20f2f7b17 already in progress: creating tablet
W20260927 23:29:03.448633 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet b373bde60f734d049c4a45255e2b2e2a because tablet deleting was already in progress. No further retry: Already present: State transition of tablet b373bde60f734d049c4a45255e2b2e2a already in progress: creating tablet
I20260927 23:29:03.449251 5553 ts_tablet_manager.cc:1905] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:29:03.454106 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
W20260927 23:29:03.454389 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 309465f419c24549bf5416a4503b9299
W20260927 23:29:03.454568 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet df6ea36da9f6455bb399c2229dcb3790
W20260927 23:29:03.454747 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:03.455214 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
W20260927 23:29:03.455616 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
W20260927 23:29:03.455793 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:29:03.455956 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
W20260927 23:29:03.456113 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
W20260927 23:29:03.456274 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:29:03.456681 5580 consensus_queue.cc:260] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:03.456456 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
W20260927 23:29:03.457616 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
W20260927 23:29:03.457829 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet e56378a5d42e47d395c881a0898da495
W20260927 23:29:03.457990 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
W20260927 23:29:03.458135 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:29:03.458295 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
W20260927 23:29:03.458453 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
W20260927 23:29:03.458628 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:29:03.458807 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:29:03.458961 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
W20260927 23:29:03.460031 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:29:03.460216 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
W20260927 23:29:03.460407 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
W20260927 23:29:03.460568 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
W20260927 23:29:03.460776 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
I20260927 23:29:03.464520 5639 raft_consensus.cc:2258] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:03.461725 5566 tablet_bootstrap.cc:492] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:03.474946 5580 ts_tablet_manager.cc:1428] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.049s user 0.005s sys 0.003s
I20260927 23:29:03.474952 5566 ts_tablet_manager.cc:1397] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.205s user 0.011s sys 0.001s
I20260927 23:29:03.475957 5580 tablet_bootstrap.cc:492] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:03.476126 4966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5073ab2faa0042569b11193952cce2ac" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:03.476835 4966 raft_consensus.cc:2454] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:03.477943 5566 raft_consensus.cc:348] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.481113 5532 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.481613 5532 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.483505 5532 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:03.485392 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:29:03.486910 5229 tablet_bootstrap.cc:492] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:03.487351 5229 ts_tablet_manager.cc:1397] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.096s user 0.007s sys 0.003s
I20260927 23:29:03.487727 5566 raft_consensus.cc:374] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:03.489800 5229 raft_consensus.cc:348] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.490522 5229 raft_consensus.cc:374] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:03.490759 5229 raft_consensus.cc:729] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.491366 5229 consensus_queue.cc:260] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.490841 5566 raft_consensus.cc:729] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.490175 5639 ts_tablet_manager.cc:1905] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:03.492949 5095 tablet_service.cc:1514] Processing DeleteTablet for tablet a338854f83f7461e8a1c790b31ef4a06 with delete_type TABLET_DATA_DELETED (replaced by tablet 1572bbcfebce4d218b1a0f38ecab0384) from {username='slave'} at 127.0.0.1:58244
I20260927 23:29:03.493147 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ae6ef92e96465fa5a414ada1b4ff51 with delete_type TABLET_DATA_DELETED (replaced by tablet b9f07e8a37404636bda765146b20fc60) from {username='slave'} at 127.0.0.1:58244
I20260927 23:29:03.494865 5566 consensus_queue.cc:260] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.499066 5054 leader_election.cc:302] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
W20260927 23:29:03.502027 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet a338854f83f7461e8a1c790b31ef4a06 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a338854f83f7461e8a1c790b31ef4a06 already in progress: creating tablet
W20260927 23:29:03.502731 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet a1ae6ef92e96465fa5a414ada1b4ff51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1ae6ef92e96465fa5a414ada1b4ff51 already in progress: creating tablet
I20260927 23:29:03.505903 5580 tablet_bootstrap.cc:654] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:03.515000 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
I20260927 23:29:03.519604 5534 raft_consensus.cc:2781] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:03.520022 5534 raft_consensus.cc:482] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:03.520358 5534 raft_consensus.cc:3037] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:29:03.523553 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:03.525370 5229 ts_tablet_manager.cc:1428] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.038s user 0.001s sys 0.004s
I20260927 23:29:03.526340 5229 tablet_bootstrap.cc:492] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:03.527484 5534 raft_consensus.cc:504] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:03.520031 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:29:03.529866 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:03.532130 5566 ts_tablet_manager.cc:1428] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.043s user 0.006s sys 0.000s
I20260927 23:29:03.534049 5510 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:03.535952 5534 leader_election.cc:288] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:03.536134 5051 leader_election.cc:341] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac
I20260927 23:29:03.539693 4967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5073ab2faa0042569b11193952cce2ac" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:03.540393 4967 raft_consensus.cc:3037] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:03.541975 5229 tablet_bootstrap.cc:654] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:03.545070 5621 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.545567 5621 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:03.547423 5621 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:03.551553 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26c0a9168b8b44fb9283c97a4206b79b" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:03.552204 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:29:03.552531 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
W20260927 23:29:03.552703 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:29:03.552870 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
W20260927 23:29:03.553043 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
W20260927 23:29:03.553210 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
I20260927 23:29:03.552994 4966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26c0a9168b8b44fb9283c97a4206b79b" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:29:03.553373 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
W20260927 23:29:03.553612 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
W20260927 23:29:03.553797 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:29:03.553957 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet b4ec12a496904c3599bdc001476ebee9
W20260927 23:29:03.554133 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
W20260927 23:29:03.555449 4769 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:29:03.556023 4769 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:03.556358 4769 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:03.557796 5621 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:03.563980 5580 tablet_bootstrap.cc:492] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:03.564522 5580 ts_tablet_manager.cc:1397] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.089s user 0.012s sys 0.000s
I20260927 23:29:03.567122 5580 raft_consensus.cc:348] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:03.567889 5580 raft_consensus.cc:374] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:03.561805 5553 ts_tablet_manager.cc:1918] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:03.568851 5553 log.cc:1196] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/a338854f83f7461e8a1c790b31ef4a06
W20260927 23:29:03.569308 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
I20260927 23:29:03.569655 5553 ts_tablet_manager.cc:1939] T a338854f83f7461e8a1c790b31ef4a06 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
W20260927 23:29:03.570237 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
W20260927 23:29:03.570541 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
I20260927 23:29:03.568169 5580 raft_consensus.cc:729] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.572209 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet a338854f83f7461e8a1c790b31ef4a06 (table default.table_77e2daeb57dc4c1d9e46b4e05c32c6f2 [id=aea1156e91a84d018912c856e3f89d34]) successfully deleted
I20260927 23:29:03.576346 4967 raft_consensus.cc:2454] T 5073ab2faa0042569b11193952cce2ac P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:03.577061 5580 consensus_queue.cc:260] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
W20260927 23:29:03.579488 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet b4ec12a496904c3599bdc001476ebee9
W20260927 23:29:03.581583 4617 catalog_manager.cc:4821] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): delete failed for tablet a1ae6ef92e96465fa5a414ada1b4ff51 because tablet deleting was already in progress. No further retry: Already present: State transition of tablet a1ae6ef92e96465fa5a414ada1b4ff51 already in progress: creating tablet
I20260927 23:29:03.586740 5054 leader_election.cc:302] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:03.586957 4625 catalog_manager.cc:2148] created HMS catalog entry for table [id=983540517b0b4405b79b922358e1c3bc]
I20260927 23:29:03.588624 5534 raft_consensus.cc:2781] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:03.589080 5534 raft_consensus.cc:686] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:03.589820 5534 consensus_queue.cc:237] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.592813 5580 ts_tablet_manager.cc:1428] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.028s user 0.006s sys 0.001s
I20260927 23:29:03.603350 5519 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.603826 5519 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.605593 5519 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:03.609998 4625 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35136 (request call id 82) took 2027 ms. Trace:
W20260927 23:29:03.610395 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:29:03.612282 5580 tablet_bootstrap.cc:492] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:03.617726 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 309465f419c24549bf5416a4503b9299 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:03.618418 5519 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.618914 5519 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.620070 5229 tablet_bootstrap.cc:492] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:03.620523 5621 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.620733 5229 ts_tablet_manager.cc:1397] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.095s user 0.008s sys 0.003s
I20260927 23:29:03.620949 5621 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.622179 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "309465f419c24549bf5416a4503b9299" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:03.622792 5621 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:03.623333 5229 raft_consensus.cc:348] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.623980 5229 raft_consensus.cc:374] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:03.624850 5229 raft_consensus.cc:729] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.625541 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b373bde60f734d049c4a45255e2b2e2a" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:03.627348 4769 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:29:03.627871 4769 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:03.628180 4769 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:03.632366 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet bdecd389ff0341f4ab867eb160786ea9 from {username='slave'} at 127.0.0.1:35136
W20260927 23:29:03.623910 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Illegal state: Tablet not RUNNING: STOPPED
I20260927 23:29:03.610417 4625 rpcz_store.cc:276] 0927 23:29:01.582541 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:01.596645 (+ 14104us) service_pool.cc:227] Handling call
0927 23:29:02.609948 (+1013303us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:29:02.611745 (+ 1797us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:03.586914 (+975169us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:03.609258 (+ 22344us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:29:03.609603 (+ 345us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:29:03.609917 (+ 314us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:03.591577 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:03.592059 (+ 482us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:03.592087 (+ 28us) write_op.cc:435] Acquired schema lock
0927 23:29:03.592100 (+ 13us) tablet.cc:660] Decoding operations
0927 23:29:03.592493 (+ 393us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:03.592545 (+ 52us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:03.592572 (+ 27us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:29:03.592813 (+ 241us) tablet.cc:699] Row locks acquired
0927 23:29:03.592824 (+ 11us) write_op.cc:260] PREPARE: finished
0927 23:29:03.592966 (+ 142us) write_op.cc:270] Start()
0927 23:29:03.593079 (+ 113us) write_op.cc:276] Timestamp: P: 1790551743592896 usec, L: 0
0927 23:29:03.593094 (+ 15us) op_driver.cc:331] REPLICATION: starting
0927 23:29:03.593724 (+ 630us) log.cc:842] Serialized 433 byte log entry
0927 23:29:03.606921 (+ 13197us) op_driver.cc:444] REPLICATION: finished
0927 23:29:03.607294 (+ 373us) write_op.cc:301] APPLY: starting
0927 23:29:03.607353 (+ 59us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:03.607469 (+ 116us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:03.607568 (+ 99us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:03.607944 (+ 376us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:03.608104 (+ 160us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:03.608144 (+ 40us) write_op.cc:312] APPLY: finished
0927 23:29:03.608374 (+ 230us) log.cc:842] Serialized 29 byte log entry
0927 23:29:03.608532 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:03.608621 (+ 89us) write_op.cc:454] Released schema lock
0927 23:29:03.608654 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:03.608849 (+ 195us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:03.608874 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":351711,"Hive Metastore.run_cpu_time_us":666,"Hive Metastore.run_wall_time_us":622300,"child_traces":[["op",{"apply.queue_time_us":196,"apply.run_cpu_time_us":1798,"apply.run_wall_time_us":1798,"num_ops":2,"prepare.queue_time_us":2879,"prepare.run_cpu_time_us":2486,"prepare.run_wall_time_us":2486,"raft.queue_time_us":202,"raft.run_cpu_time_us":636,"raft.run_wall_time_us":635,"replication_time_us":13739}]]}
I20260927 23:29:03.635462 5580 tablet_bootstrap.cc:654] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:03.636548 5519 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:03.640643 5229 consensus_queue.cc:260] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.642966 5229 ts_tablet_manager.cc:1428] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.022s user 0.001s sys 0.004s
I20260927 23:29:03.643873 5229 tablet_bootstrap.cc:492] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
W20260927 23:29:03.640681 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 54040.5us
I20260927 23:29:03.649281 5519 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.649887 5519 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.652626 5621 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:03.658097 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:29:03.660565 5519 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:03.687325 5532 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.687759 5532 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.689728 5532 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:03.691701 5229 tablet_bootstrap.cc:654] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:03.692510 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:29:03.694852 4625 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:29:03.695111 4625 catalog_manager.cc:5328] Ignoring report from unknown tablet c739222645654eb39285a2ad1b6c35f5
W20260927 23:29:03.695277 4625 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
I20260927 23:29:03.710852 5118 raft_consensus.cc:3037] T 48dcf44bb7b744b9a98626dbfb6f4e6c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:29:03.710960 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:29:03.718550 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
W20260927 23:29:03.720471 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:29:03.720914 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:03.721859 5519 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:03.722332 5534 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.723347 5534 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:03.726902 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:29:03.727772 5534 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:03.732502 5534 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:03.735285 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:29:03.735746 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
W20260927 23:29:03.736960 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:29:03.737884 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:03.737403 5534 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:03.739184 5437 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:03.739840 5437 consensus_queue.cc:1035] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:03.743889 5229 tablet_bootstrap.cc:492] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:03.744351 5229 ts_tablet_manager.cc:1397] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.101s user 0.003s sys 0.008s
I20260927 23:29:03.746455 5639 ts_tablet_manager.cc:1918] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
W20260927 23:29:03.747056 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:29:03.747432 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:03.748185 5437 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:03.757535 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:03.758989 5639 log.cc:1196] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/309465f419c24549bf5416a4503b9299
I20260927 23:29:03.759791 5639 ts_tablet_manager.cc:1939] T 309465f419c24549bf5416a4503b9299 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
W20260927 23:29:03.763621 4625 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
I20260927 23:29:03.765954 5639 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53202 (request call id 113) took 1095 ms. Trace:
I20260927 23:29:03.766340 5639 rpcz_store.cc:276] 0927 23:29:02.670109 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:02.670245 (+ 136us) service_pool.cc:227] Handling call
0927 23:29:03.443013 (+772768us) ts_tablet_manager.cc:1145] Deleting tablet 309465f419c24549bf5416a4503b9299
0927 23:29:03.443062 (+ 49us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:29:03.656545 (+213483us) tablet_metadata.cc:641] Metadata flushed
0927 23:29:03.746145 (+ 89600us) tablet_metadata.cc:641] Metadata flushed
0927 23:29:03.765878 (+ 19733us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":772199}
I20260927 23:29:03.769447 5229 raft_consensus.cc:348] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.770298 5229 raft_consensus.cc:374] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:03.770541 5229 raft_consensus.cc:729] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.771070 5229 consensus_queue.cc:260] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.774173 5580 tablet_bootstrap.cc:492] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:03.775484 5580 ts_tablet_manager.cc:1397] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.163s user 0.005s sys 0.007s
I20260927 23:29:03.778277 5229 ts_tablet_manager.cc:1428] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.034s user 0.004s sys 0.001s
I20260927 23:29:03.778546 5580 raft_consensus.cc:348] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.779158 5229 tablet_bootstrap.cc:492] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:03.779309 5580 raft_consensus.cc:374] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:03.779646 5580 raft_consensus.cc:729] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:03.780341 5580 consensus_queue.cc:260] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.782752 5580 ts_tablet_manager.cc:1428] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260927 23:29:03.782939 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:03.784093 5229 tablet_bootstrap.cc:654] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:03.796672 4625 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:03.796988 4625 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:29:03.797190 4625 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
W20260927 23:29:03.797384 4625 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
W20260927 23:29:03.797564 4625 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
I20260927 23:29:03.797821 5639 tablet_replica.cc:330] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:03.800740 5639 raft_consensus.cc:2229] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:03.802995 5639 raft_consensus.cc:2258] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:03.818391 5639 ts_tablet_manager.cc:1905] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:03.835775 4626 master_service.cc:946] ReplaceTablet: received request to replace tablet 26c0a9168b8b44fb9283c97a4206b79b from {username='slave'} at 127.0.0.1:35136
W20260927 23:29:03.857349 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
W20260927 23:29:03.857792 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
I20260927 23:29:03.885213 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ae6ef92e96465fa5a414ada1b4ff51 with delete_type TABLET_DATA_DELETED (replaced by tablet b9f07e8a37404636bda765146b20fc60) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:03.885085 5519 raft_consensus.cc:482] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.885641 5519 raft_consensus.cc:504] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.887012 4625 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_1fbfdf99ff4e4165bfcf02ceb0247c87"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:29:03.889811 5519 leader_election.cc:288] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:03.890894 5051 leader_election.cc:341] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790
I20260927 23:29:03.894109 5510 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.897909 4622 catalog_manager.cc:2148] created HMS catalog entry for table [id=70012e8fafc049d2833983e702dd05ba]
I20260927 23:29:03.894541 5510 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.899992 5510 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:03.919664 4622 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35136 (request call id 85) took 2101 ms. Trace:
I20260927 23:29:03.920018 4622 rpcz_store.cc:276] 0927 23:29:01.818179 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:01.818391 (+ 212us) service_pool.cc:227] Handling call
0927 23:29:02.617587 (+799196us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:29:02.619176 (+ 1589us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:03.897860 (+1278684us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:03.918953 (+ 21093us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:29:03.919288 (+ 335us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:29:03.919586 (+ 298us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:03.899882 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:03.900399 (+ 517us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:03.900429 (+ 30us) write_op.cc:435] Acquired schema lock
0927 23:29:03.900445 (+ 16us) tablet.cc:660] Decoding operations
0927 23:29:03.900792 (+ 347us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:03.900835 (+ 43us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:03.900863 (+ 28us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:29:03.901086 (+ 223us) tablet.cc:699] Row locks acquired
0927 23:29:03.901102 (+ 16us) write_op.cc:260] PREPARE: finished
0927 23:29:03.901234 (+ 132us) write_op.cc:270] Start()
0927 23:29:03.901332 (+ 98us) write_op.cc:276] Timestamp: P: 1790551743901191 usec, L: 0
0927 23:29:03.901351 (+ 19us) op_driver.cc:331] REPLICATION: starting
0927 23:29:03.901927 (+ 576us) log.cc:842] Serialized 433 byte log entry
0927 23:29:03.916020 (+ 14093us) op_driver.cc:444] REPLICATION: finished
0927 23:29:03.916511 (+ 491us) write_op.cc:301] APPLY: starting
0927 23:29:03.916576 (+ 65us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:03.916690 (+ 114us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:03.916777 (+ 87us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:03.917184 (+ 407us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:03.917335 (+ 151us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:03.917375 (+ 40us) write_op.cc:312] APPLY: finished
0927 23:29:03.917575 (+ 200us) log.cc:842] Serialized 29 byte log entry
0927 23:29:03.917753 (+ 178us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:03.917830 (+ 77us) write_op.cc:454] Released schema lock
0927 23:29:03.917863 (+ 33us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:03.918038 (+ 175us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:03.918059 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":966767,"Hive Metastore.run_cpu_time_us":636,"Hive Metastore.run_wall_time_us":311348,"child_traces":[["op",{"apply.queue_time_us":322,"apply.run_cpu_time_us":1723,"apply.run_wall_time_us":1723,"num_ops":2,"prepare.queue_time_us":311,"prepare.run_cpu_time_us":2334,"prepare.run_wall_time_us":3261,"raft.queue_time_us":90,"raft.run_cpu_time_us":626,"raft.run_wall_time_us":627,"replication_time_us":14584}]]}
W20260927 23:29:03.930285 4616 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b373bde60f734d049c4a45255e2b2e2a (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b373bde60f734d049c4a45255e2b2e2a consensus state has no leader
I20260927 23:29:03.931926 4968 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df6ea36da9f6455bb399c2229dcb3790" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:03.932704 4968 raft_consensus.cc:2454] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:03.933288 5553 tablet_replica.cc:330] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:03.934105 5553 raft_consensus.cc:2229] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:03.935782 5553 raft_consensus.cc:2258] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:03.940402 5553 ts_tablet_manager.cc:1905] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:29:03.943468 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet b373bde60f734d049c4a45255e2b2e2a
W20260927 23:29:03.904187 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:29:03.942435 5054 leader_election.cc:302] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
W20260927 23:29:03.947630 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
W20260927 23:29:03.947917 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
W20260927 23:29:03.948150 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
W20260927 23:29:03.949673 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:03.952661 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
I20260927 23:29:03.956059 5681 raft_consensus.cc:482] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:03.956553 5681 raft_consensus.cc:504] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:03.958765 5681 leader_election.cc:288] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:03.961534 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289d65ad2eb64788a6fa7dbc1b48122c" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:03.963039 4910 leader_election.cc:341] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:03.963078 4833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289d65ad2eb64788a6fa7dbc1b48122c" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:03.963671 4833 raft_consensus.cc:2454] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:03.965337 4907 leader_election.cc:302] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters: f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:03.968343 5681 raft_consensus.cc:2781] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:03.968726 5681 raft_consensus.cc:482] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:03.969090 5681 raft_consensus.cc:3037] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:03.984973 4626 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Sending DeleteTablet for 3 replicas of tablet 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:29:03.987480 4812 tablet_service.cc:1514] Processing DeleteTablet for tablet 26c0a9168b8b44fb9283c97a4206b79b with delete_type TABLET_DATA_DELETED (replaced by tablet 935c9a2e4d6c416db02443d2aaa57752) from {username='slave'} at 127.0.0.1:53202
I20260927 23:29:03.988125 5229 tablet_bootstrap.cc:492] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:03.988565 5229 ts_tablet_manager.cc:1397] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.210s user 0.045s sys 0.049s
I20260927 23:29:03.988924 4626 catalog_manager.cc:6565] ReplaceTablet: tablet 26c0a9168b8b44fb9283c97a4206b79b deleted and replaced by tablet 935c9a2e4d6c416db02443d2aaa57752
I20260927 23:29:03.996803 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 26c0a9168b8b44fb9283c97a4206b79b with delete_type TABLET_DATA_DELETED (replaced by tablet 935c9a2e4d6c416db02443d2aaa57752) from {username='slave'} at 127.0.0.1:34620
I20260927 23:29:04.002970 4812 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0aaa4d2fc8491aaf77eda6aa94616f (DEFAULT_TABLE table=default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.003273 5639 ts_tablet_manager.cc:1918] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:04.003679 5639 log.cc:1196] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:04.003914 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0aaa4d2fc8491aaf77eda6aa94616f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.004288 5639 ts_tablet_manager.cc:1939] T 1766d227273e4c7592c4f0a20f2f7b17 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:04.005061 5639 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53202 (request call id 115) took 1013 ms. Trace:
I20260927 23:29:04.005339 5639 rpcz_store.cc:276] 0927 23:29:02.992153 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:02.992257 (+ 104us) service_pool.cc:227] Handling call
0927 23:29:03.797309 (+805052us) ts_tablet_manager.cc:1145] Deleting tablet 1766d227273e4c7592c4f0a20f2f7b17
0927 23:29:03.797418 (+ 109us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:29:03.905954 (+108536us) tablet_metadata.cc:641] Metadata flushed
0927 23:29:04.003033 (+ 97079us) tablet_metadata.cc:641] Metadata flushed
0927 23:29:04.005011 (+ 1978us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":804465}
I20260927 23:29:04.006392 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 1766d227273e4c7592c4f0a20f2f7b17 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:29:04.010972 5553 ts_tablet_manager.cc:1918] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:04.011721 5639 tablet_replica.cc:330] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:04.012590 5639 raft_consensus.cc:2229] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:04.017869 5639 raft_consensus.cc:2258] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:04.018329 4626 master_service.cc:946] ReplaceTablet: received request to replace tablet a1ae6ef92e96465fa5a414ada1b4ff51 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:04.020370 5639 ts_tablet_manager.cc:1905] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:04.021697 5553 log.cc:1196] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:29:04.021749 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 74330us
I20260927 23:29:04.022821 5510 raft_consensus.cc:2781] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:04.022475 5553 ts_tablet_manager.cc:1939] T a1ae6ef92e96465fa5a414ada1b4ff51 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:04.023528 5229 raft_consensus.cc:348] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.023138 5510 raft_consensus.cc:482] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:04.024170 5229 raft_consensus.cc:374] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.024396 5510 raft_consensus.cc:3037] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:04.024478 5229 raft_consensus.cc:729] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:04.025068 5229 consensus_queue.cc:260] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.026979 5229 ts_tablet_manager.cc:1428] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.038s user 0.005s sys 0.000s
I20260927 23:29:04.027750 5229 tablet_bootstrap.cc:492] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
W20260927 23:29:04.022349 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:29:04.028810 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:04.029884 5435 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:04.028832 5527 consensus_queue.cc:1035] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:29:04.031061 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet 26c0a9168b8b44fb9283c97a4206b79b with delete_type TABLET_DATA_DELETED (replaced by tablet 935c9a2e4d6c416db02443d2aaa57752) from {username='slave'} at 127.0.0.1:58244
I20260927 23:29:04.031673 4953 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0aaa4d2fc8491aaf77eda6aa94616f (DEFAULT_TABLE table=default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.032675 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0aaa4d2fc8491aaf77eda6aa94616f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.000088 5681 raft_consensus.cc:504] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.037633 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 8546d6e754784ff886f421eb0ddb796a (DEFAULT_TABLE table=default.table_1c3db2bbcc0e488ea4b6637492ff4221 [id=09f1820cca0c454e928a02f0677e7c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.038617 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8546d6e754784ff886f421eb0ddb796a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.038648 4833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289d65ad2eb64788a6fa7dbc1b48122c" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:04.039229 4833 raft_consensus.cc:3037] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:04.047379 4833 raft_consensus.cc:2454] T 289d65ad2eb64788a6fa7dbc1b48122c P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:04.049660 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet a1ae6ef92e96465fa5a414ada1b4ff51 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:29:04.055827 5553 tablet_replica.cc:330] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:04.059229 5095 tablet_service.cc:1467] Processing CreateTablet for tablet 0e0aaa4d2fc8491aaf77eda6aa94616f (DEFAULT_TABLE table=default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc]), partition=RANGE (key) PARTITION UNBOUNDED
W20260927 23:29:04.060207 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
I20260927 23:29:04.060349 5095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e0aaa4d2fc8491aaf77eda6aa94616f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:29:04.060588 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:29:04.060773 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:29:04.055905 4769 consensus_peers.cc:487] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca -> Peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Couldn't send request to peer f5ec98b2972d44838fa273eb7a10d01a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:04.069720 5553 raft_consensus.cc:2229] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:04.069860 5229 tablet_bootstrap.cc:654] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.071933 5093 tablet_service.cc:1514] Processing DeleteTablet for tablet a338854f83f7461e8a1c790b31ef4a06 with delete_type TABLET_DATA_DELETED (replaced by tablet 1572bbcfebce4d218b1a0f38ecab0384) from {username='slave'} at 127.0.0.1:58244
I20260927 23:29:04.071841 5098 tablet_service.cc:1467] Processing CreateTablet for tablet 8546d6e754784ff886f421eb0ddb796a (DEFAULT_TABLE table=default.table_1c3db2bbcc0e488ea4b6637492ff4221 [id=09f1820cca0c454e928a02f0677e7c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.072923 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8546d6e754784ff886f421eb0ddb796a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.076462 4813 tablet_service.cc:1467] Processing CreateTablet for tablet 1572bbcfebce4d218b1a0f38ecab0384 (DEFAULT_TABLE table=default.table_77e2daeb57dc4c1d9e46b4e05c32c6f2 [id=aea1156e91a84d018912c856e3f89d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.076781 5553 raft_consensus.cc:2258] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:04.062503 5681 leader_election.cc:288] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:04.077411 4813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1572bbcfebce4d218b1a0f38ecab0384. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:29:04.080462 4617 catalog_manager.cc:4821] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): delete failed for tablet 26c0a9168b8b44fb9283c97a4206b79b because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 26c0a9168b8b44fb9283c97a4206b79b already in progress: creating tablet
I20260927 23:29:04.081713 5699 tablet_replica.cc:330] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a: stopping tablet replica
I20260927 23:29:04.082499 5510 raft_consensus.cc:504] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.082912 4907 leader_election.cc:302] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:04.083074 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "289d65ad2eb64788a6fa7dbc1b48122c" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:04.087782 5701 raft_consensus.cc:2781] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
W20260927 23:29:04.087770 4910 leader_election.cc:341] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:04.088970 4952 tablet_service.cc:1467] Processing CreateTablet for tablet 8546d6e754784ff886f421eb0ddb796a (DEFAULT_TABLE table=default.table_1c3db2bbcc0e488ea4b6637492ff4221 [id=09f1820cca0c454e928a02f0677e7c9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.089159 5553 ts_tablet_manager.cc:1905] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:04.093741 5532 consensus_queue.cc:1035] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:29:04.094594 4626 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
I20260927 23:29:04.088285 5701 raft_consensus.cc:686] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
I20260927 23:29:04.095395 5701 consensus_queue.cc:237] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.097359 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 1572bbcfebce4d218b1a0f38ecab0384 (DEFAULT_TABLE table=default.table_77e2daeb57dc4c1d9e46b4e05c32c6f2 [id=aea1156e91a84d018912c856e3f89d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.098357 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1572bbcfebce4d218b1a0f38ecab0384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.101477 4811 tablet_service.cc:1467] Processing CreateTablet for tablet 1c89dab9709541c1afc6a8cdcd43a118 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.102459 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c89dab9709541c1afc6a8cdcd43a118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.098400 4954 tablet_service.cc:1467] Processing CreateTablet for tablet 1572bbcfebce4d218b1a0f38ecab0384 (DEFAULT_TABLE table=default.table_77e2daeb57dc4c1d9e46b4e05c32c6f2 [id=aea1156e91a84d018912c856e3f89d34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.105213 5580 tablet_bootstrap.cc:492] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:04.106369 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1572bbcfebce4d218b1a0f38ecab0384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.109833 5580 tablet_bootstrap.cc:654] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.109965 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8546d6e754784ff886f421eb0ddb796a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.112689 5699 raft_consensus.cc:2229] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:04.113335 5699 raft_consensus.cc:2258] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:04.115406 5699 ts_tablet_manager.cc:1905] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:04.120862 5229 tablet_bootstrap.cc:492] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:04.121255 5229 ts_tablet_manager.cc:1397] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.094s user 0.008s sys 0.002s
I20260927 23:29:04.123430 5229 raft_consensus.cc:348] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.124089 5229 raft_consensus.cc:374] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.124362 5229 raft_consensus.cc:729] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:04.124886 5229 consensus_queue.cc:260] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.126721 5229 ts_tablet_manager.cc:1428] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260927 23:29:04.130450 5229 tablet_bootstrap.cc:492] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:04.142889 5229 tablet_bootstrap.cc:654] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.148633 4626 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_85c5c1d72c47496b94eb0f8c080ead12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:29:04.149372 5229 tablet_bootstrap.cc:492] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:04.149747 5229 ts_tablet_manager.cc:1397] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.000s
I20260927 23:29:04.151918 5229 raft_consensus.cc:348] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.152573 5229 raft_consensus.cc:374] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.152806 5229 raft_consensus.cc:729] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:04.153456 5229 consensus_queue.cc:260] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:04.159943 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:04.161322 5229 ts_tablet_manager.cc:1428] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20260927 23:29:04.162717 5229 tablet_bootstrap.cc:492] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:04.178649 5229 tablet_bootstrap.cc:654] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.178993 4953 tablet_service.cc:1467] Processing CreateTablet for tablet 1c89dab9709541c1afc6a8cdcd43a118 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.179941 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c89dab9709541c1afc6a8cdcd43a118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.187836 5532 consensus_queue.cc:1035] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:29:04.189179 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:04.200814 5639 ts_tablet_manager.cc:1918] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:04.203158 5639 log.cc:1196] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:04.204674 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet 9dffe7e40ecb40c68a06f0cf43861777 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:04.208046 5639 ts_tablet_manager.cc:1939] T b373bde60f734d049c4a45255e2b2e2a P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:04.211299 5639 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53202 (request call id 116) took 1218 ms. Trace:
W20260927 23:29:04.214910 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:04.216199 5639 rpcz_store.cc:276] 0927 23:29:02.992472 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:02.992901 (+ 429us) service_pool.cc:227] Handling call
0927 23:29:04.011568 (+1018667us) ts_tablet_manager.cc:1145] Deleting tablet b373bde60f734d049c4a45255e2b2e2a
0927 23:29:04.011609 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:29:04.131120 (+119511us) tablet_metadata.cc:641] Metadata flushed
0927 23:29:04.200568 (+ 69448us) tablet_metadata.cc:641] Metadata flushed
0927 23:29:04.211240 (+ 10672us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1018181}
I20260927 23:29:04.218621 5510 leader_election.cc:288] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:04.220108 5051 leader_election.cc:341] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790
I20260927 23:29:04.221653 5639 tablet_replica.cc:330] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:04.221494 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet b373bde60f734d049c4a45255e2b2e2a (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:29:04.226723 5699 ts_tablet_manager.cc:1918] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:04.228943 5553 ts_tablet_manager.cc:1918] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:04.223814 5639 raft_consensus.cc:2229] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:04.229988 5639 raft_consensus.cc:2258] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:04.230484 5553 log.cc:1196] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:29:04.234814 4811 tablet_service.cc:1467] Processing CreateTablet for tablet b9f07e8a37404636bda765146b20fc60 (DEFAULT_TABLE table=default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.235747 4951 tablet_service.cc:1467] Processing CreateTablet for tablet b9f07e8a37404636bda765146b20fc60 (DEFAULT_TABLE table=default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.237010 4951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9f07e8a37404636bda765146b20fc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.232730 5553 ts_tablet_manager.cc:1939] T 26c0a9168b8b44fb9283c97a4206b79b P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:04.247349 5699 log.cc:1196] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/a338854f83f7461e8a1c790b31ef4a06
I20260927 23:29:04.250730 5702 tablet_bootstrap.cc:492] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:04.254053 5621 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:04.254494 5621 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.259881 5229 tablet_bootstrap.cc:492] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:04.260283 5229 ts_tablet_manager.cc:1397] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.098s user 0.032s sys 0.030s
I20260927 23:29:04.261214 5532 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:04.261607 5532 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.262486 5229 raft_consensus.cc:348] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.263099 5229 raft_consensus.cc:374] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.263402 5532 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:04.263763 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
W20260927 23:29:04.263978 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
W20260927 23:29:04.264132 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
W20260927 23:29:04.264956 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 75575.5us
I20260927 23:29:04.263413 5229 raft_consensus.cc:729] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:04.267212 5534 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:04.267397 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 1c89dab9709541c1afc6a8cdcd43a118 (DEFAULT_TABLE table=default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.267872 5534 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.268397 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c89dab9709541c1afc6a8cdcd43a118. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.269693 5534 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:04.271342 4769 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:04.271914 5621 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:04.273408 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ae6ef92e96465fa5a414ada1b4ff51" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:04.275578 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:04.278735 5699 ts_tablet_manager.cc:1939] T a338854f83f7461e8a1c790b31ef4a06 P f5ec98b2972d44838fa273eb7a10d01a: Deleting consensus metadata
I20260927 23:29:04.282269 5639 ts_tablet_manager.cc:1905] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:29:04.282745 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:04.282894 5621 consensus_queue.cc:1035] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:29:04.287976 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:29:04.291008 5580 tablet_bootstrap.cc:492] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:04.291386 5580 ts_tablet_manager.cc:1397] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.186s user 0.008s sys 0.014s
I20260927 23:29:04.291411 5532 consensus_queue.cc:1035] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:04.292536 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9f07e8a37404636bda765146b20fc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.267843 5229 consensus_queue.cc:260] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.297705 5118 raft_consensus.cc:2454] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 0.
I20260927 23:29:04.300915 5229 ts_tablet_manager.cc:1428] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.040s user 0.005s sys 0.001s
I20260927 23:29:04.302626 4769 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a; no voters: bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:04.303747 5621 raft_consensus.cc:2781] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:04.304116 5621 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:04.304502 5621 raft_consensus.cc:3037] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:04.308274 5580 raft_consensus.cc:348] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.308943 5580 raft_consensus.cc:374] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.309177 5580 raft_consensus.cc:729] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:04.309718 5580 consensus_queue.cc:260] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.312449 5229 tablet_bootstrap.cc:492] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
W20260927 23:29:04.313206 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
W20260927 23:29:04.313478 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
W20260927 23:29:04.313764 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
W20260927 23:29:04.313975 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:04.314184 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
W20260927 23:29:04.314419 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
W20260927 23:29:04.314618 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
I20260927 23:29:04.326879 5229 tablet_bootstrap.cc:654] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.330570 5580 ts_tablet_manager.cc:1428] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.039s user 0.007s sys 0.000s
I20260927 23:29:04.331444 5580 tablet_bootstrap.cc:492] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:04.332108 4627 master_service.cc:946] ReplaceTablet: received request to replace tablet bdecd389ff0341f4ab867eb160786ea9 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:04.340361 5702 tablet_bootstrap.cc:654] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:04.341653 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 50355.2us
I20260927 23:29:04.344129 4617 catalog_manager.cc:4841] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): tablet a338854f83f7461e8a1c790b31ef4a06 (table default.table_77e2daeb57dc4c1d9e46b4e05c32c6f2 [id=aea1156e91a84d018912c856e3f89d34]) successfully deleted
I20260927 23:29:04.345433 4617 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet 26c0a9168b8b44fb9283c97a4206b79b (table default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]) successfully deleted
I20260927 23:29:04.346161 4623 catalog_manager.cc:2148] created HMS catalog entry for table [id=078228a392424e6c96b3e5dbc1a7380d]
W20260927 23:29:04.352723 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:29:04.359403 5095 tablet_service.cc:1467] Processing CreateTablet for tablet b9f07e8a37404636bda765146b20fc60 (DEFAULT_TABLE table=default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.360517 5095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9f07e8a37404636bda765146b20fc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.366015 5580 tablet_bootstrap.cc:654] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.370879 5621 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.373473 4623 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35136 (request call id 84) took 2559 ms. Trace:
I20260927 23:29:04.374809 4967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df6ea36da9f6455bb399c2229dcb3790" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:04.375396 4967 raft_consensus.cc:3037] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:04.373795 4623 rpcz_store.cc:276] 0927 23:29:01.814583 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:01.814708 (+ 125us) service_pool.cc:227] Handling call
0927 23:29:02.615402 (+800694us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:29:02.625946 (+ 10544us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:04.346125 (+1720179us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:04.372800 (+ 26675us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:29:04.373124 (+ 324us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:29:04.373420 (+ 296us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:04.347839 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:04.348243 (+ 404us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:04.348268 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:29:04.348284 (+ 16us) tablet.cc:660] Decoding operations
0927 23:29:04.348612 (+ 328us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:04.348675 (+ 63us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:04.348700 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:29:04.348919 (+ 219us) tablet.cc:699] Row locks acquired
0927 23:29:04.348935 (+ 16us) write_op.cc:260] PREPARE: finished
0927 23:29:04.349072 (+ 137us) write_op.cc:270] Start()
0927 23:29:04.349160 (+ 88us) write_op.cc:276] Timestamp: P: 1790551744348967 usec, L: 0
0927 23:29:04.349176 (+ 16us) op_driver.cc:331] REPLICATION: starting
0927 23:29:04.349664 (+ 488us) log.cc:842] Serialized 433 byte log entry
0927 23:29:04.370589 (+ 20925us) op_driver.cc:444] REPLICATION: finished
0927 23:29:04.371436 (+ 847us) write_op.cc:301] APPLY: starting
0927 23:29:04.371491 (+ 55us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:04.371592 (+ 101us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:04.371688 (+ 96us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:04.372041 (+ 353us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:04.372166 (+ 125us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:04.372205 (+ 39us) write_op.cc:312] APPLY: finished
0927 23:29:04.372391 (+ 186us) log.cc:842] Serialized 29 byte log entry
0927 23:29:04.372522 (+ 131us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.372587 (+ 65us) write_op.cc:454] Released schema lock
0927 23:29:04.372617 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:04.372774 (+ 157us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.372806 (+ 32us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":1271394,"Hive Metastore.run_cpu_time_us":597,"Hive Metastore.run_wall_time_us":448148,"child_traces":[["op",{"apply.queue_time_us":695,"apply.run_cpu_time_us":1518,"apply.run_wall_time_us":1518,"num_ops":2,"prepare.queue_time_us":101,"prepare.run_cpu_time_us":2131,"prepare.run_wall_time_us":3813,"raft.queue_time_us":90,"raft.run_cpu_time_us":594,"raft.run_wall_time_us":594,"replication_time_us":21331,"spinlock_wait_cycles":2304}]]}
W20260927 23:29:04.409118 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:29:04.409693 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
W20260927 23:29:04.410785 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:04.411962 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:04.388900 5510 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:04.416532 4953 tablet_service.cc:1467] Processing CreateTablet for tablet 340a88a1702d48be9a8373fb3c6d540c (DEFAULT_TABLE table=default.table_829e227cbcac4bc585663258666cd43a [id=70012e8fafc049d2833983e702dd05ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.417522 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 340a88a1702d48be9a8373fb3c6d540c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.418335 4951 tablet_service.cc:1467] Processing CreateTablet for tablet 935c9a2e4d6c416db02443d2aaa57752 (DEFAULT_TABLE table=default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.421829 4951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 935c9a2e4d6c416db02443d2aaa57752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.406303 4811 tablet_service.cc:1467] Processing CreateTablet for tablet 340a88a1702d48be9a8373fb3c6d540c (DEFAULT_TABLE table=default.table_829e227cbcac4bc585663258666cd43a [id=70012e8fafc049d2833983e702dd05ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.428550 5527 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:04.410452 4812 tablet_service.cc:1467] Processing CreateTablet for tablet 935c9a2e4d6c416db02443d2aaa57752 (DEFAULT_TABLE table=default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.435530 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 340a88a1702d48be9a8373fb3c6d540c. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:29:04.413812 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:04.436272 4812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 935c9a2e4d6c416db02443d2aaa57752. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:29:04.436281 5051 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.1:40571, user_credentials={real_user=slave}} blocked reactor thread for 43882.1us
I20260927 23:29:04.413666 5519 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:04.434194 5510 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.446282 5510 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:04.460431 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:04.468705 5621 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Requested vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:04.425448 4967 raft_consensus.cc:2454] T df6ea36da9f6455bb399c2229dcb3790 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:04.478925 5229 tablet_bootstrap.cc:492] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:04.479429 5229 ts_tablet_manager.cc:1397] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.167s user 0.011s sys 0.002s
W20260927 23:29:04.480618 5717 log.cc:924] Time spent T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: Append to log took a long time: real 0.051s user 0.006s sys 0.000s
W20260927 23:29:04.495823 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:04.492488 5229 raft_consensus.cc:348] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:04.501701 5229 raft_consensus.cc:374] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.502720 5229 raft_consensus.cc:729] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:04.504015 5229 consensus_queue.cc:260] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
W20260927 23:29:04.507620 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 92455.2us
W20260927 23:29:04.508322 4769 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 110724us
I20260927 23:29:04.534215 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ae6ef92e96465fa5a414ada1b4ff51" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:04.534798 5118 raft_consensus.cc:3037] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:04.535306 5229 ts_tablet_manager.cc:1428] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.056s user 0.005s sys 0.000s
I20260927 23:29:04.535075 4954 tablet_service.cc:1467] Processing CreateTablet for tablet be4adc02302b44c2b20a2be202ebe68d (DEFAULT_TABLE table=default.table_8a2ddb6dcbbc4b8bba970c3bf5c37848 [id=078228a392424e6c96b3e5dbc1a7380d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.536069 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be4adc02302b44c2b20a2be202ebe68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.536099 5229 tablet_bootstrap.cc:492] T 289d65ad2eb64788a6fa7dbc1b48122c P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:04.537014 4810 tablet_service.cc:1467] Processing CreateTablet for tablet be4adc02302b44c2b20a2be202ebe68d (DEFAULT_TABLE table=default.table_8a2ddb6dcbbc4b8bba970c3bf5c37848 [id=078228a392424e6c96b3e5dbc1a7380d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.538009 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be4adc02302b44c2b20a2be202ebe68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.550904 5702 tablet_bootstrap.cc:492] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
W20260927 23:29:04.495227 5712 log.cc:924] Time spent T 6c3c55f368dd4ba991daded55d2e3069 P bad92a10de2c4459827cdbb18854d76d: Append to log took a long time: real 0.067s user 0.000s sys 0.000s
W20260927 23:29:04.554662 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 46497.5us
I20260927 23:29:04.555713 5054 leader_election.cc:302] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
W20260927 23:29:04.560926 5731 log.cc:924] Time spent T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a: Append to log took a long time: real 0.055s user 0.000s sys 0.000s
I20260927 23:29:04.566650 5229 tablet_bootstrap.cc:654] T 289d65ad2eb64788a6fa7dbc1b48122c P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.568167 5532 consensus_queue.cc:1035] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:04.569988 5118 raft_consensus.cc:2454] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 1.
I20260927 23:29:04.576053 5721 raft_consensus.cc:2781] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:04.576516 5721 raft_consensus.cc:686] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:04.577227 5721 consensus_queue.cc:237] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:04.578877 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:04.579102 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
W20260927 23:29:04.579280 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
W20260927 23:29:04.579470 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
W20260927 23:29:04.579686 4627 catalog_manager.cc:5328] Ignoring report from unknown tablet a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:04.585729 5097 tablet_service.cc:1467] Processing CreateTablet for tablet 935c9a2e4d6c416db02443d2aaa57752 (DEFAULT_TABLE table=default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.585731 5095 tablet_service.cc:1467] Processing CreateTablet for tablet 340a88a1702d48be9a8373fb3c6d540c (DEFAULT_TABLE table=default.table_829e227cbcac4bc585663258666cd43a [id=70012e8fafc049d2833983e702dd05ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.586742 5097 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 935c9a2e4d6c416db02443d2aaa57752. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.587307 5095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 340a88a1702d48be9a8373fb3c6d540c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.580969 4769 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:04.591275 5098 tablet_service.cc:1467] Processing CreateTablet for tablet be4adc02302b44c2b20a2be202ebe68d (DEFAULT_TABLE table=default.table_8a2ddb6dcbbc4b8bba970c3bf5c37848 [id=078228a392424e6c96b3e5dbc1a7380d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:04.592223 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be4adc02302b44c2b20a2be202ebe68d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:04.594298 4627 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35136:
name: "default.Table_e2a4264ee2df4a97a99f0846c2489d0b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:29:04.595528 5702 ts_tablet_manager.cc:1397] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.345s user 0.012s sys 0.004s
I20260927 23:29:04.596743 5580 tablet_bootstrap.cc:492] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:04.597132 5580 ts_tablet_manager.cc:1397] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.266s user 0.010s sys 0.010s
I20260927 23:29:04.596050 5621 raft_consensus.cc:2781] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:04.607491 5702 raft_consensus.cc:348] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.608613 5702 raft_consensus.cc:374] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.611186 5580 raft_consensus.cc:348] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.611838 5580 raft_consensus.cc:374] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.612077 5580 raft_consensus.cc:729] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
W20260927 23:29:04.612216 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:04.612617 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:04.613346 5718 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:04.625059 5527 consensus_queue.cc:1035] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:04.627858 5527 consensus_queue.cc:1035] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:29:04.596627 4769 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:04.632534 5580 consensus_queue.cc:260] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.617316 5621 raft_consensus.cc:686] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: Becoming Leader. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Running, Role: LEADER
I20260927 23:29:04.634271 5621 consensus_queue.cc:237] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.636704 5702 raft_consensus.cc:729] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:04.637774 5698 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:29:04.638262 5698 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:04.637439 5702 consensus_queue.cc:260] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.644550 5639 ts_tablet_manager.cc:1918] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:04.647670 5639 log.cc:1196] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/a338854f83f7461e8a1c790b31ef4a06
I20260927 23:29:04.647455 5604 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 83cd22a89b7846999c0e4dfab80ebe9d. Latest consensus state: current_term: 1 leader_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:04.650357 5639 ts_tablet_manager.cc:1939] T a338854f83f7461e8a1c790b31ef4a06 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:04.651067 5511 rpcz_store.cc:275] Call kudu.master.MasterService.GetMasterRegistration from 127.0.0.1:47144 (request call id 0) took 1433 ms. Trace:
I20260927 23:29:04.655015 5639 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53202 (request call id 118) took 1434 ms. Trace:
I20260927 23:29:04.655695 5580 ts_tablet_manager.cc:1428] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.058s user 0.001s sys 0.005s
I20260927 23:29:04.652374 5702 ts_tablet_manager.cc:1428] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.055s user 0.005s sys 0.001s
I20260927 23:29:04.656545 5580 tablet_bootstrap.cc:492] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:04.656823 5702 tablet_bootstrap.cc:492] T 8546d6e754784ff886f421eb0ddb796a P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:04.661134 4561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 114 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:29:04.655334 5639 rpcz_store.cc:276] 0927 23:29:03.220806 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:03.220909 (+ 103us) service_pool.cc:227] Handling call
0927 23:29:04.221491 (+1000582us) ts_tablet_manager.cc:1145] Deleting tablet a338854f83f7461e8a1c790b31ef4a06
0927 23:29:04.221532 (+ 41us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:29:04.499775 (+278243us) tablet_metadata.cc:641] Metadata flushed
0927 23:29:04.588130 (+ 88355us) tablet_metadata.cc:641] Metadata flushed
0927 23:29:04.654958 (+ 66828us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1000043}
I20260927 23:29:04.663995 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet a338854f83f7461e8a1c790b31ef4a06 (table default.table_77e2daeb57dc4c1d9e46b4e05c32c6f2 [id=aea1156e91a84d018912c856e3f89d34]) successfully deleted
I20260927 23:29:04.664623 5639 tablet_replica.cc:330] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
W20260927 23:29:04.666554 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
W20260927 23:29:04.666854 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
W20260927 23:29:04.667070 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
I20260927 23:29:04.670908 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 114 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:29:04.672607 5474 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, f8460f73b40c41a28ec020fc04203564
W20260927 23:29:04.673388 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:04.674670 5698 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:29:04.675657 5698 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:29:04.680246 5702 tablet_bootstrap.cc:654] T 8546d6e754784ff886f421eb0ddb796a P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.648201 5604 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:29:04.675190 5523 rpcz_store.cc:275] Call kudu.master.MasterService.ListTables from 127.0.0.1:47128 (request call id 14) took 1553 ms. Trace:
I20260927 23:29:04.687209 5523 rpcz_store.cc:276] 0927 23:29:03.111410 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:03.213247 (+101837us) service_pool.cc:227] Handling call
0927 23:29:04.663519 (+1450272us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400010754. stack: 00007ffb50419033 00007ffb50418f30 00007ffb4ff9a534 00000000004db2b4 00000000004db249 00007ffb57325af0 00007ffb573a8f2a 00007ffb59274d6c 00007ffb59370c56 00007ffb56ebb137 00007ffb56ebb0c2 00007ffb56ebaff1 00007ffb56ebaf6c 00007ffb56eba222 00007ffb5514f94c 00007ffb5514ed86
0927 23:29:04.664252 (+ 733us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3278878592}
I20260927 23:29:04.651432 5511 rpcz_store.cc:276] 0927 23:29:03.217620 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:03.217804 (+ 184us) service_pool.cc:227] Handling call
0927 23:29:04.649690 (+1431886us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b5400010754. stack: 00007ffb50419033 00007ffb50418f30 00007ffb4ff9a534 00000000004db2b4 00000000004db249 00007ffb57325af0 00007ffb573a8f2a 00007ffb59274d6c 00007ffb59371c3a 00007ffb56edbe37 00007ffb56edbdc2 00007ffb56edbcf1 00007ffb56edbc6c 00007ffb56edaf22 00007ffb5514f94c 00007ffb5514ed86
0927 23:29:04.650880 (+ 1190us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3263886080}
I20260927 23:29:04.682778 5516 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" instance_seqno: 1790551727089681) as {username='slave'} at 127.2.57.1:41989; Asking this server to re-register.
I20260927 23:29:04.689570 5229 tablet_bootstrap.cc:492] T 289d65ad2eb64788a6fa7dbc1b48122c P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:04.689963 5229 ts_tablet_manager.cc:1397] T 289d65ad2eb64788a6fa7dbc1b48122c P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.154s user 0.013s sys 0.002s
W20260927 23:29:04.689870 5516 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.1:41989 (request call id 9) took 1497 ms (client timeout 1000 ms). Trace:
W20260927 23:29:04.690228 5516 rpcz_store.cc:269] 0927 23:29:03.192962 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:03.193623 (+ 661us) service_pool.cc:227] Handling call
0927 23:29:04.645389 (+1451766us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b5400010754. stack: 00007ffb50419033 00007ffb50418f30 00007ffb4ff9a534 00000000004db2b4 00000000004db249 00007ffb57325af0 00007ffb573a8f2a 00007ffb59274d6c 00007ffb5936ec3e 00007ffb56ea5fb4 00007ffb56ea5f42 00007ffb56ea5e71 00007ffb56ea5dec 00007ffb56ea50a2 00007ffb5514f94c 00007ffb5514ed86
0927 23:29:04.689756 (+ 44367us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3314463104}
I20260927 23:29:04.683540 5493 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" instance_seqno: 1790551729034875) as {username='slave'} at 127.2.57.2:45045; Asking this server to re-register.
I20260927 23:29:04.684158 5504 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" instance_seqno: 1790551729034875) as {username='slave'} at 127.2.57.2:45045; Asking this server to re-register.
W20260927 23:29:04.692708 5504 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.2:45045 (request call id 6) took 1380 ms (client timeout 1000 ms). Trace:
W20260927 23:29:04.693027 5504 rpcz_store.cc:269] 0927 23:29:03.312727 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:03.313044 (+ 317us) service_pool.cc:227] Handling call
0927 23:29:04.664893 (+1351849us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400010754. stack: 00007ffb50419033 00007ffb50418f30 00007ffb4ff9a534 00000000004db2b4 00000000004db249 00007ffb57325af0 00007ffb573a8f2a 00007ffb59274d6c 00007ffb5936ec3e 00007ffb56ea5fb4 00007ffb56ea5f42 00007ffb56ea5e71 00007ffb56ea5dec 00007ffb56ea50a2 00007ffb5514f94c 00007ffb5514ed86
0927 23:29:04.692648 (+ 27755us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3040329600}
I20260927 23:29:04.684906 5496 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" instance_seqno: 1790551731011766) as {username='slave'} at 127.2.57.3:45329; Asking this server to re-register.
I20260927 23:29:04.694554 5021 heartbeater.cc:464] Registering TS with master...
I20260927 23:29:04.685518 5499 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" instance_seqno: 1790551727089681) as {username='slave'} at 127.2.57.1:41989; Asking this server to re-register.
I20260927 23:29:04.698315 5522 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" instance_seqno: 1790551731011766) as {username='slave'} at 127.2.57.3:45329; Asking this server to re-register.
I20260927 23:29:04.699096 5165 heartbeater.cc:464] Registering TS with master...
I20260927 23:29:04.692075 5229 raft_consensus.cc:348] T 289d65ad2eb64788a6fa7dbc1b48122c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.705062 5229 raft_consensus.cc:374] T 289d65ad2eb64788a6fa7dbc1b48122c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.705327 5229 raft_consensus.cc:729] T 289d65ad2eb64788a6fa7dbc1b48122c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
W20260927 23:29:04.707398 5522 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:45329 (request call id 2) took 1526 ms (client timeout 1000 ms). Trace:
I20260927 23:29:04.707749 5229 consensus_queue.cc:260] T 289d65ad2eb64788a6fa7dbc1b48122c P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:04.708344 5522 rpcz_store.cc:269] 0927 23:29:03.181023 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:03.181216 (+ 193us) service_pool.cc:227] Handling call
0927 23:29:04.641662 (+1460446us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b5400010754. stack: 00007ffb50419033 00007ffb50418f30 00007ffb4ff9a534 00000000004db2b4 00000000004db249 00007ffb57325af0 00007ffb573a8f2a 00007ffb59274d6c 00007ffb5936ec3e 00007ffb56ea5fb4 00007ffb56ea5f42 00007ffb56ea5e71 00007ffb56ea5dec 00007ffb56ea50a2 00007ffb5514f94c 00007ffb5514ed86
0927 23:29:04.707334 (+ 65672us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3331909376}
I20260927 23:29:04.711701 5229 ts_tablet_manager.cc:1428] T 289d65ad2eb64788a6fa7dbc1b48122c P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260927 23:29:04.712525 5229 tablet_bootstrap.cc:492] T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:04.721001 5702 tablet_bootstrap.cc:492] T 8546d6e754784ff886f421eb0ddb796a P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:04.721571 5702 ts_tablet_manager.cc:1397] T 8546d6e754784ff886f421eb0ddb796a P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.065s user 0.008s sys 0.008s
I20260927 23:29:04.724466 5580 tablet_bootstrap.cc:654] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.729843 5229 tablet_bootstrap.cc:654] T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.724620 5702 raft_consensus.cc:348] T 8546d6e754784ff886f421eb0ddb796a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.732115 5702 raft_consensus.cc:374] T 8546d6e754784ff886f421eb0ddb796a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.733240 5702 raft_consensus.cc:729] T 8546d6e754784ff886f421eb0ddb796a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:04.735016 5702 consensus_queue.cc:260] T 8546d6e754784ff886f421eb0ddb796a P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.739874 5702 ts_tablet_manager.cc:1428] T 8546d6e754784ff886f421eb0ddb796a P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.018s user 0.005s sys 0.000s
I20260927 23:29:04.743047 5702 tablet_bootstrap.cc:492] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:04.754135 5580 tablet_bootstrap.cc:492] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:04.754665 5580 ts_tablet_manager.cc:1397] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.098s user 0.007s sys 0.004s
I20260927 23:29:04.757488 5702 tablet_bootstrap.cc:654] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.766501 4623 master_service.cc:946] ReplaceTablet: received request to replace tablet 26c0a9168b8b44fb9283c97a4206b79b from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:04.766780 5229 tablet_bootstrap.cc:492] T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:04.767167 5229 ts_tablet_manager.cc:1397] T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.055s user 0.006s sys 0.004s
I20260927 23:29:04.767493 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d90eb9269884933a9af575b5af8a47e from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:04.769294 5229 raft_consensus.cc:348] T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.769904 5229 raft_consensus.cc:374] T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.770125 5229 raft_consensus.cc:729] T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:04.757081 5580 raft_consensus.cc:348] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:04.774485 5580 raft_consensus.cc:374] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.778808 5580 raft_consensus.cc:729] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:04.779310 5229 consensus_queue.cc:260] T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.781837 5580 consensus_queue.cc:260] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:04.787264 5580 ts_tablet_manager.cc:1428] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.032s user 0.005s sys 0.000s
I20260927 23:29:04.787797 5681 raft_consensus.cc:482] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:04.788226 5681 raft_consensus.cc:504] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.790225 5681 leader_election.cc:288] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:04.793072 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8546d6e754784ff886f421eb0ddb796a" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:04.793673 4830 raft_consensus.cc:2454] T 8546d6e754784ff886f421eb0ddb796a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:04.794946 4907 leader_election.cc:302] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
W20260927 23:29:04.801282 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
W20260927 23:29:04.801658 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
W20260927 23:29:04.802399 5474 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.60:33749, user_credentials={real_user=slave}} blocked reactor thread for 129987us
I20260927 23:29:04.806016 4880 heartbeater.cc:464] Registering TS with master...
I20260927 23:29:04.806329 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8546d6e754784ff886f421eb0ddb796a" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:04.806890 5118 raft_consensus.cc:2454] T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:04.808534 5681 raft_consensus.cc:2781] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:04.808917 5681 raft_consensus.cc:482] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:04.809264 5681 raft_consensus.cc:3037] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:04.810425 5522 ts_manager.cc:194] Registered new tserver with Master: bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:04.811748 5499 ts_manager.cc:194] Registered new tserver with Master: f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:04.822799 5544 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47136 (request call id 0) took 1885 ms. Trace:
I20260927 23:29:04.824066 5639 raft_consensus.cc:2229] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:29:04.825644 5639 pending_rounds.cc:62] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca: Trying to abort 1 pending ops.
I20260927 23:29:04.825861 5639 pending_rounds.cc:68] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334099946035785728 op_type: NO_OP noop_request { }
I20260927 23:29:04.826215 5639 raft_consensus.cc:2866] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260927 23:29:04.826532 5639 raft_consensus.cc:2258] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 23:29:04.831061 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:04.833810 5496 ts_manager.cc:194] Registered new tserver with Master: 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
I20260927 23:29:04.839433 5437 raft_consensus.cc:482] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:04.839854 5437 raft_consensus.cc:504] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.842052 5437 leader_election.cc:288] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:04.842370 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9cb9be216d42fa81948f5d9da3d4d8" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
W20260927 23:29:04.842746 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 215123us
I20260927 23:29:04.843109 5229 ts_tablet_manager.cc:1428] T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.076s user 0.006s sys 0.001s
I20260927 23:29:04.846095 5639 ts_tablet_manager.cc:1905] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:04.857349 5702 tablet_bootstrap.cc:492] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:04.848747 5580 tablet_bootstrap.cc:492] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:04.861471 5664 catalog_manager.cc:1259] Loaded cluster ID: 6dc11a7b873d4b11830ae46041eb6de8
I20260927 23:29:04.861852 5664 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: loading cluster ID for follower catalog manager: success
I20260927 23:29:04.862813 5229 tablet_bootstrap.cc:492] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:04.837796 5681 raft_consensus.cc:504] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.869839 5702 ts_tablet_manager.cc:1397] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.127s user 0.005s sys 0.007s
I20260927 23:29:04.876886 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8546d6e754784ff886f421eb0ddb796a" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:04.877461 5118 raft_consensus.cc:3037] T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:04.874531 5229 tablet_bootstrap.cc:654] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.884135 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe9cb9be216d42fa81948f5d9da3d4d8" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
W20260927 23:29:04.884613 5717 log.cc:924] Time spent T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: Append to log took a long time: real 0.062s user 0.001s sys 0.000s
W20260927 23:29:04.887336 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
W20260927 23:29:04.887632 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet df6ea36da9f6455bb399c2229dcb3790
W20260927 23:29:04.887892 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:29:04.888126 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
W20260927 23:29:04.888399 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:29:04.888644 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
W20260927 23:29:04.888882 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
W20260927 23:29:04.889122 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:29:04.889367 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
I20260927 23:29:04.889604 5118 raft_consensus.cc:2454] T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:04.890580 4910 leader_election.cc:302] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:04.891444 5700 raft_consensus.cc:2781] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:04.891848 5700 raft_consensus.cc:686] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
I20260927 23:29:04.892598 5700 consensus_queue.cc:237] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.896510 5580 tablet_bootstrap.cc:654] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:04.890523 5681 leader_election.cc:288] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
W20260927 23:29:04.907657 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
I20260927 23:29:04.875106 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8546d6e754784ff886f421eb0ddb796a" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:04.914222 4830 raft_consensus.cc:3037] T 8546d6e754784ff886f421eb0ddb796a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:04.922286 4830 raft_consensus.cc:2454] T 8546d6e754784ff886f421eb0ddb796a P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:04.926900 5229 tablet_bootstrap.cc:492] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:04.929421 5580 tablet_bootstrap.cc:492] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:04.929876 5580 ts_tablet_manager.cc:1397] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.081s user 0.006s sys 0.003s
I20260927 23:29:04.930323 5229 ts_tablet_manager.cc:1397] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.068s user 0.023s sys 0.008s
I20260927 23:29:04.933545 5639 ts_tablet_manager.cc:1918] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:29:04.934603 5229 raft_consensus.cc:348] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.935247 5229 raft_consensus.cc:374] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.935461 5229 raft_consensus.cc:729] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:04.936064 5229 consensus_queue.cc:260] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.941660 4622 catalog_manager.cc:5495] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d reported cstate change: term changed from 0 to 1, leader changed from <none> to bad92a10de2c4459827cdbb18854d76d (127.2.57.2). New cstate: current_term: 1 leader_uuid: "bad92a10de2c4459827cdbb18854d76d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260927 23:29:04.948372 5229 ts_tablet_manager.cc:1428] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260927 23:29:04.950096 5229 tablet_bootstrap.cc:492] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:04.955005 5229 tablet_bootstrap.cc:654] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:04.966743 4616 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b373bde60f734d049c4a45255e2b2e2a (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b373bde60f734d049c4a45255e2b2e2a consensus state has no leader
I20260927 23:29:04.979265 5664 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: acquiring CA information for follower catalog manager: success
I20260927 23:29:04.980866 5580 raft_consensus.cc:348] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.981575 5580 raft_consensus.cc:374] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:04.981832 5580 raft_consensus.cc:729] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:04.982420 5580 consensus_queue.cc:260] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:04.992965 5054 leader_election.cc:302] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:04.993997 5639 log.cc:1196] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:04.995330 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet bdecd389ff0341f4ab867eb160786ea9 from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:04.916096 5702 raft_consensus.cc:348] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:05.000278 5702 raft_consensus.cc:374] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:05.000687 5702 raft_consensus.cc:729] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:05.001500 5702 consensus_queue.cc:260] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
W20260927 23:29:05.002526 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
I20260927 23:29:05.003403 4622 catalog_manager.cc:5587] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717) does not have the latest schema for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]). Expected version 1 got 0
W20260927 23:29:05.005725 4622 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet b9f07e8a37404636bda765146b20fc60 consensus state has no leader
I20260927 23:29:05.011620 4623 master_service.cc:946] ReplaceTablet: received request to replace tablet 0d90eb9269884933a9af575b5af8a47e from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:05.015401 5702 ts_tablet_manager.cc:1428] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.101s user 0.005s sys 0.000s
I20260927 23:29:05.023391 5437 raft_consensus.cc:2726] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:05.030120 5118 raft_consensus.cc:3037] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:05.036859 5229 tablet_bootstrap.cc:492] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:05.037333 5229 ts_tablet_manager.cc:1397] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.087s user 0.007s sys 0.008s
I20260927 23:29:05.042054 5229 raft_consensus.cc:348] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:05.042762 5229 raft_consensus.cc:374] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:05.042991 5229 raft_consensus.cc:729] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:05.043505 5229 consensus_queue.cc:260] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:05.048825 5702 tablet_bootstrap.cc:492] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:05.061334 5229 ts_tablet_manager.cc:1428] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.024s user 0.004s sys 0.002s
W20260927 23:29:05.059170 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 66277.5us
I20260927 23:29:05.017166 5639 ts_tablet_manager.cc:1939] T a1ae6ef92e96465fa5a414ada1b4ff51 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:05.085911 5664 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 23:29:05.097968 5639 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53202 (request call id 119) took 1703 ms. Trace:
I20260927 23:29:05.099153 5580 ts_tablet_manager.cc:1428] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.169s user 0.005s sys 0.000s
I20260927 23:29:05.101166 5580 tablet_bootstrap.cc:492] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:29:05.107383 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
I20260927 23:29:05.107964 5639 rpcz_store.cc:276] 0927 23:29:03.394879 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:03.395026 (+ 147us) service_pool.cc:227] Handling call
0927 23:29:04.664460 (+1269434us) ts_tablet_manager.cc:1145] Deleting tablet a1ae6ef92e96465fa5a414ada1b4ff51
0927 23:29:04.664508 (+ 48us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:29:04.825196 (+160688us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54000a4e54. stack: 00007f501ca0e033 00007f501ca0df30 00007f501c58f534 00000000004db2b4 00000000004db249 00007f502391a471 00007f502399d352 00007f5023db326c 00007f50249c0cfa 00007f50249f0e10 00007f50249d7e0d 00007f50249d7db9 00007f50249d7d49 00007f50249d7d11 00007f50249d6f3d 0000000000580784
0927 23:29:04.875381 (+ 50185us) tablet_metadata.cc:641] Metadata flushed
0927 23:29:04.933304 (+ 57923us) tablet_metadata.cc:641] Metadata flushed
0927 23:29:05.097898 (+164594us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":354358144,"tablet-delete.queue_time_us":1268845}
I20260927 23:29:05.112953 5700 raft_consensus.cc:482] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:05.113494 5700 raft_consensus.cc:504] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.115460 5700 leader_election.cc:288] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:05.117456 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0aaa4d2fc8491aaf77eda6aa94616f" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:05.118105 5115 raft_consensus.cc:2454] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:05.119299 4910 leader_election.cc:302] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:05.117931 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0aaa4d2fc8491aaf77eda6aa94616f" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:05.120108 5700 raft_consensus.cc:2781] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:05.120435 5700 raft_consensus.cc:482] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:05.120464 4830 raft_consensus.cc:2454] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:05.120721 5700 raft_consensus.cc:3037] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:05.122789 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet a1ae6ef92e96465fa5a414ada1b4ff51 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:29:05.124723 5639 tablet_replica.cc:330] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:05.125564 5229 tablet_bootstrap.cc:492] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:05.125523 5639 raft_consensus.cc:2229] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:05.125985 5639 raft_consensus.cc:2258] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:05.126463 4622 catalog_manager.cc:5587] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717) does not have the latest schema for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]). Expected version 1 got 0
I20260927 23:29:05.144141 5700 raft_consensus.cc:504] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.151973 5700 leader_election.cc:288] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:05.147619 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0aaa4d2fc8491aaf77eda6aa94616f" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:05.152972 4830 raft_consensus.cc:3037] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:05.154326 5639 ts_tablet_manager.cc:1905] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:05.157333 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e0aaa4d2fc8491aaf77eda6aa94616f" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:05.157878 5115 raft_consensus.cc:3037] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:05.164228 4830 raft_consensus.cc:2454] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:05.164343 5115 raft_consensus.cc:2454] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:05.166400 4910 leader_election.cc:302] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
W20260927 23:29:05.167711 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
I20260927 23:29:05.182485 5681 raft_consensus.cc:2781] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:05.183070 5681 raft_consensus.cc:686] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
I20260927 23:29:05.183881 5681 consensus_queue.cc:237] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.186012 5118 raft_consensus.cc:3037] T 289d65ad2eb64788a6fa7dbc1b48122c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:05.187444 5229 tablet_bootstrap.cc:654] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:05.189370 5580 tablet_bootstrap.cc:654] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:05.195744 5744 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:29:05.196169 5744 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:05.197888 5744 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:29:05.199513 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 114 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:29:05.202024 4561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 114 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:29:05.203531 5471 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:05.205915 5744 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:29:05.211223 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:29:05.211690 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
W20260927 23:29:05.212067 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
W20260927 23:29:05.212342 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
I20260927 23:29:05.212581 5229 tablet_bootstrap.cc:492] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
W20260927 23:29:05.212698 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
W20260927 23:29:05.212982 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
I20260927 23:29:05.213068 5229 ts_tablet_manager.cc:1397] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.088s user 0.006s sys 0.004s
W20260927 23:29:05.213202 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:29:05.213671 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
I20260927 23:29:05.216229 5229 raft_consensus.cc:348] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.216889 5229 raft_consensus.cc:374] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260927 23:29:05.217219 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
I20260927 23:29:05.217139 5229 raft_consensus.cc:729] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
W20260927 23:29:05.217486 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
W20260927 23:29:05.217726 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
I20260927 23:29:05.217914 5229 consensus_queue.cc:260] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:05.218700 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
W20260927 23:29:05.218945 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
W20260927 23:29:05.220244 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
W20260927 23:29:05.221397 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
I20260927 23:29:05.224951 5580 tablet_bootstrap.cc:492] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:05.225364 5580 ts_tablet_manager.cc:1397] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.124s user 0.005s sys 0.004s
I20260927 23:29:05.227523 5580 raft_consensus.cc:348] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.228945 5229 ts_tablet_manager.cc:1428] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260927 23:29:05.229295 5700 consensus_queue.cc:1035] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:05.232882 5229 tablet_bootstrap.cc:492] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:05.247331 5229 tablet_bootstrap.cc:654] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:05.265868 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
W20260927 23:29:05.266204 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
I20260927 23:29:05.267539 4623 catalog_manager.cc:5587] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519) does not have the latest schema for tablet 1c89dab9709541c1afc6a8cdcd43a118 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]). Expected version 1 got 0
I20260927 23:29:05.228193 5580 raft_consensus.cc:374] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:05.268594 5580 raft_consensus.cc:729] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:05.269423 5580 consensus_queue.cc:260] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.276266 5580 ts_tablet_manager.cc:1428] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.051s user 0.006s sys 0.000s
W20260927 23:29:05.286623 4623 catalog_manager.cc:5687] Failed to send AlterTable RPC for tablet 1c89dab9709541c1afc6a8cdcd43a118 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1): Not found: failed to reset TS proxy: No leader found: Tablet 1c89dab9709541c1afc6a8cdcd43a118 consensus state has no leader
I20260927 23:29:05.291635 5527 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:05.292160 5527 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:29:05.294126 5527 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:29:05.296605 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:29:05.299094 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
I20260927 23:29:05.151306 5702 tablet_bootstrap.cc:654] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:05.308362 4622 catalog_manager.cc:5495] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d reported cstate change: term changed from 0 to 1, leader changed from <none> to bad92a10de2c4459827cdbb18854d76d (127.2.57.2). New cstate: current_term: 1 leader_uuid: "bad92a10de2c4459827cdbb18854d76d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260927 23:29:05.316396 5527 raft_consensus.cc:482] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:05.316924 5527 raft_consensus.cc:504] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.320338 5527 leader_election.cc:288] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:05.322307 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c89dab9709541c1afc6a8cdcd43a118" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:05.323028 4972 raft_consensus.cc:2454] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:05.323135 5681 raft_consensus.cc:482] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:05.323628 5681 raft_consensus.cc:504] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.324539 5054 leader_election.cc:302] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:05.325424 5527 raft_consensus.cc:2781] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:05.326599 5527 raft_consensus.cc:482] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:05.327024 5527 raft_consensus.cc:3037] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:05.327416 5681 leader_election.cc:288] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:05.326905 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f07e8a37404636bda765146b20fc60" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:05.332211 4910 leader_election.cc:341] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:05.336615 5701 raft_consensus.cc:482] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:05.337121 5701 raft_consensus.cc:504] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:05.339130 5701 leader_election.cc:288] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
I20260927 23:29:05.345033 5527 raft_consensus.cc:504] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.346961 5527 leader_election.cc:288] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:05.348039 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c89dab9709541c1afc6a8cdcd43a118" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:05.348768 4972 raft_consensus.cc:3037] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:05.355017 5580 tablet_bootstrap.cc:492] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:05.355161 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1572bbcfebce4d218b1a0f38ecab0384" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:05.356256 5115 raft_consensus.cc:2454] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:05.357542 4910 leader_election.cc:302] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:05.358549 5701 raft_consensus.cc:2781] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:05.358919 5701 raft_consensus.cc:482] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:05.359300 5701 raft_consensus.cc:3037] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:05.367902 5701 raft_consensus.cc:504] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:05.372931 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1572bbcfebce4d218b1a0f38ecab0384" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:05.373652 5115 raft_consensus.cc:3037] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:05.369971 4972 raft_consensus.cc:2454] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:05.379982 5054 leader_election.cc:302] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:05.381285 5437 raft_consensus.cc:2781] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:05.381731 5437 raft_consensus.cc:686] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:05.382520 5437 consensus_queue.cc:237] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.384104 5718 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:05.384531 5718 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:05.386397 5718 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:05.394593 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:29:05.400758 5701 leader_election.cc:288] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:29:05.403384 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
I20260927 23:29:05.405905 4623 catalog_manager.cc:5587] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717) does not have the latest schema for tablet 1c89dab9709541c1afc6a8cdcd43a118 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]). Expected version 1 got 0
I20260927 23:29:05.412622 4623 catalog_manager.cc:5495] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a reported cstate change: term changed from 0 to 1, leader changed from <none> to f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3). New cstate: current_term: 1 leader_uuid: "f5ec98b2972d44838fa273eb7a10d01a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } health_report { overall_health: HEALTHY } } }
I20260927 23:29:05.417068 4623 catalog_manager.cc:5587] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519) does not have the latest schema for tablet 1c89dab9709541c1afc6a8cdcd43a118 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]). Expected version 1 got 0
I20260927 23:29:05.423327 5115 raft_consensus.cc:2454] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:05.425607 4910 leader_election.cc:302] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:05.426805 5701 raft_consensus.cc:2781] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:05.428942 5701 raft_consensus.cc:686] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
I20260927 23:29:05.430231 5701 consensus_queue.cc:237] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:05.445973 5681 consensus_queue.cc:1035] T 289d65ad2eb64788a6fa7dbc1b48122c P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260927 23:29:05.448266 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:29:05.448742 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:05.448501 4832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1572bbcfebce4d218b1a0f38ecab0384" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:05.449282 4832 raft_consensus.cc:3037] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:05.450582 4833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1572bbcfebce4d218b1a0f38ecab0384" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:05.451818 5534 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:05.453485 4829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c89dab9709541c1afc6a8cdcd43a118" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:05.454859 4833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c89dab9709541c1afc6a8cdcd43a118" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:05.471242 4971 raft_consensus.cc:1261] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Refusing update from remote peer f5ec98b2972d44838fa273eb7a10d01a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:29:05.474767 5718 consensus_queue.cc:1035] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:29:05.481860 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:29:05.485240 4829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26c0a9168b8b44fb9283c97a4206b79b" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:05.486863 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f07e8a37404636bda765146b20fc60" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
W20260927 23:29:05.515662 5051 leader_election.cc:341] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:29:05.515998 4907 leader_election.cc:341] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:05.516410 4907 leader_election.cc:302] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
W20260927 23:29:05.516650 5051 leader_election.cc:341] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:29:05.517367 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Illegal state: Tablet not RUNNING: STOPPED
I20260927 23:29:05.517688 5051 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:05.519488 5700 raft_consensus.cc:2726] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:05.520732 5527 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:05.534904 4832 raft_consensus.cc:2454] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
W20260927 23:29:05.546613 5051 consensus_peers.cc:487] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:05.552976 5527 consensus_queue.cc:1035] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:29:05.556970 5702 tablet_bootstrap.cc:492] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:05.557852 5702 ts_tablet_manager.cc:1397] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.509s user 0.013s sys 0.006s
W20260927 23:29:05.579108 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 1: this message will repeat every 5th retry.
W20260927 23:29:05.585278 5768 log.cc:924] Time spent T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a: Append to log took a long time: real 0.133s user 0.001s sys 0.000s
I20260927 23:29:05.595974 5702 raft_consensus.cc:348] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.596889 5702 raft_consensus.cc:374] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:05.597185 5702 raft_consensus.cc:729] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:05.599277 5580 tablet_bootstrap.cc:654] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:05.612432 5722 mvcc.cc:205] Tried to move back new op lower bound from 7334099949430976512 to 7334099949111496704. Current Snapshot: MvccSnapshot[applied={T|T < 7334099949430976512}]
I20260927 23:29:05.611560 5702 consensus_queue.cc:260] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.624791 5702 ts_tablet_manager.cc:1428] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.066s user 0.004s sys 0.003s
I20260927 23:29:05.543601 4623 master_service.cc:946] ReplaceTablet: received request to replace tablet 48dcf44bb7b744b9a98626dbfb6f4e6c from {username='slave'} at 127.0.0.1:35136
W20260927 23:29:05.589227 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
I20260927 23:29:05.633819 5580 tablet_bootstrap.cc:492] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:05.639190 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 48dcf44bb7b744b9a98626dbfb6f4e6c from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:05.639767 5580 ts_tablet_manager.cc:1397] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.285s user 0.004s sys 0.008s
I20260927 23:29:05.641117 5702 tablet_bootstrap.cc:492] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:05.645814 5639 ts_tablet_manager.cc:1918] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260927 23:29:05.649569 5639 log.cc:1196] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:29:05.643946 5580 raft_consensus.cc:348] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.663508 5580 raft_consensus.cc:374] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:05.663774 5580 raft_consensus.cc:729] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:05.662801 5639 ts_tablet_manager.cc:1939] T 26c0a9168b8b44fb9283c97a4206b79b P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:05.672609 5580 consensus_queue.cc:260] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.675320 5639 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53202 (request call id 120) took 1688 ms. Trace:
I20260927 23:29:05.675726 5639 rpcz_store.cc:276] 0927 23:29:03.987179 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:03.987312 (+ 133us) service_pool.cc:227] Handling call
0927 23:29:05.124563 (+1137251us) ts_tablet_manager.cc:1145] Deleting tablet 26c0a9168b8b44fb9283c97a4206b79b
0927 23:29:05.124608 (+ 45us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0927 23:29:05.524937 (+400329us) tablet_metadata.cc:641] Metadata flushed
0927 23:29:05.620165 (+ 95228us) tablet_metadata.cc:641] Metadata flushed
0927 23:29:05.675204 (+ 55039us) inbound_call.cc:173] Queueing success response
Metrics: {"tablet-delete.queue_time_us":1136683}
I20260927 23:29:05.677209 4614 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 26c0a9168b8b44fb9283c97a4206b79b (table default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]) successfully deleted
I20260927 23:29:05.682582 5580 ts_tablet_manager.cc:1428] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.043s user 0.005s sys 0.000s
I20260927 23:29:05.686414 5702 tablet_bootstrap.cc:654] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:05.687860 5580 tablet_bootstrap.cc:492] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:29:05.714332 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
I20260927 23:29:05.719292 5580 tablet_bootstrap.cc:654] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:05.722150 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
W20260927 23:29:05.723766 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet df6ea36da9f6455bb399c2229dcb3790
W20260927 23:29:05.724023 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
I20260927 23:29:05.735229 5776 tablet.cc:1722] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:29:05.737397 5774 tablet.cc:1722] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260927 23:29:05.765043 5229 thread.cc:636] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.512s user 0.023s sys 0.025s
W20260927 23:29:05.765589 5229 thread.cc:602] log-alloc [worker] (thread pool) Time spent starting thread: real 0.512s user 0.023s sys 0.025s
I20260927 23:29:05.770196 5702 tablet_bootstrap.cc:492] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:05.770680 5702 ts_tablet_manager.cc:1397] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.130s user 0.009s sys 0.012s
I20260927 23:29:05.773764 5702 raft_consensus.cc:348] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.774499 5702 raft_consensus.cc:374] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:05.774748 5702 raft_consensus.cc:729] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:05.784163 5702 consensus_queue.cc:260] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.797410 5702 ts_tablet_manager.cc:1428] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.026s user 0.007s sys 0.000s
I20260927 23:29:05.798521 5702 tablet_bootstrap.cc:492] T be4adc02302b44c2b20a2be202ebe68d P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
W20260927 23:29:05.800038 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
W20260927 23:29:05.800400 4623 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:29:05.809754 4622 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
I20260927 23:29:05.811255 4622 catalog_manager.cc:5587] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571) does not have the latest schema for tablet 1c89dab9709541c1afc6a8cdcd43a118 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]). Expected version 1 got 0
I20260927 23:29:05.813565 5229 tablet_bootstrap.cc:492] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:05.814109 5229 ts_tablet_manager.cc:1397] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.581s user 0.027s sys 0.032s
I20260927 23:29:05.858979 5700 raft_consensus.cc:482] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:05.859467 5700 raft_consensus.cc:504] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.861405 5700 leader_election.cc:288] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:05.864526 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "340a88a1702d48be9a8373fb3c6d540c" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:05.866562 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "340a88a1702d48be9a8373fb3c6d540c" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:05.867870 4907 leader_election.cc:341] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:29:05.869591 4910 leader_election.cc:341] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:05.869939 4910 leader_election.cc:302] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d; no voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:05.870955 5700 raft_consensus.cc:2726] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:05.870800 5681 consensus_queue.cc:1035] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:29:05.874089 5779 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:29:05.886067 5779 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:05.932400 4560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 114 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:29:05.932694 5779 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:29:05.967039 5681 consensus_queue.cc:1035] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:29:05.991410 5621 raft_consensus.cc:482] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:05.992033 5621 raft_consensus.cc:504] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:05.994323 5621 leader_election.cc:288] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:05.996990 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f07e8a37404636bda765146b20fc60" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:05.998636 4769 leader_election.cc:341] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:06.015096 5229 raft_consensus.cc:348] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.015795 5229 raft_consensus.cc:374] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:06.016036 5229 raft_consensus.cc:729] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:06.027202 5229 consensus_queue.cc:260] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.029354 5229 ts_tablet_manager.cc:1428] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.215s user 0.005s sys 0.000s
I20260927 23:29:06.033792 5229 tablet_bootstrap.cc:492] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:06.029983 4629 catalog_manager.cc:3411] renamed table default.table_77e2daeb57dc4c1d9e46b4e05c32c6f2 [id=aea1156e91a84d018912c856e3f89d34] in HMS: new name default.table_ecd08889b3cb4f09b30c4588cdceda4b
I20260927 23:29:06.077138 5681 consensus_queue.cc:1035] T 8546d6e754784ff886f421eb0ddb796a P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:06.086467 5534 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:06.086982 5534 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:06.089272 5534 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:06.095196 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:29:06.097939 5229 tablet_bootstrap.cc:654] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:06.104106 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 2 candidate_status { last_received { term: 1 index: 114 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:29:06.110358 5474 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:06.115000 5779 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:29:06.119921 5621 raft_consensus.cc:482] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:06.120400 5621 raft_consensus.cc:504] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.122232 5621 leader_election.cc:288] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:06.125267 4971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f07e8a37404636bda765146b20fc60" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:06.125918 4971 raft_consensus.cc:2454] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 0.
I20260927 23:29:06.135398 5580 tablet_bootstrap.cc:492] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:06.135834 5580 ts_tablet_manager.cc:1397] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.448s user 0.017s sys 0.007s
I20260927 23:29:06.142347 4623 catalog_manager.cc:5495] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d reported cstate change: term changed from 0 to 1, leader changed from <none> to bad92a10de2c4459827cdbb18854d76d (127.2.57.2). New cstate: current_term: 1 leader_uuid: "bad92a10de2c4459827cdbb18854d76d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } health_report { overall_health: HEALTHY } } }
W20260927 23:29:06.145377 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:29:06.145764 5051 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:06.147265 5534 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:06.150372 5580 raft_consensus.cc:348] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.151010 5580 raft_consensus.cc:374] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:06.151234 5580 raft_consensus.cc:729] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:06.153995 5580 consensus_queue.cc:260] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.163951 5781 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:29:06.164445 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c89dab9709541c1afc6a8cdcd43a118" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:06.164444 5781 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:06.166724 5114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c89dab9709541c1afc6a8cdcd43a118" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:06.167970 4769 leader_election.cc:302] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters: f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:06.169521 4769 leader_election.cc:302] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:06.180657 5783 raft_consensus.cc:2781] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:06.181123 5783 raft_consensus.cc:482] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:06.181519 5783 raft_consensus.cc:3037] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:06.184955 5786 raft_consensus.cc:2726] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:06.193008 5787 log.cc:924] Time spent T 8546d6e754784ff886f421eb0ddb796a P f5ec98b2972d44838fa273eb7a10d01a: Append to log took a long time: real 0.055s user 0.000s sys 0.003s
I20260927 23:29:06.194008 5229 tablet_bootstrap.cc:492] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:06.195432 5229 ts_tablet_manager.cc:1397] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.162s user 0.009s sys 0.002s
I20260927 23:29:06.200489 5783 raft_consensus.cc:504] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.197599 5229 raft_consensus.cc:348] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.208102 5229 raft_consensus.cc:374] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:06.208467 5229 raft_consensus.cc:729] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:06.210018 5781 leader_election.cc:288] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:29:06.210103 5702 tablet_bootstrap.cc:654] T be4adc02302b44c2b20a2be202ebe68d P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:06.210526 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 143 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:29:06.211300 5681 consensus_queue.cc:1035] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:06.213076 5229 consensus_queue.cc:260] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:06.214820 4687 catalog_manager.cc:5755] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 1572bbcfebce4d218b1a0f38ecab0384 consensus state has no leader
W20260927 23:29:06.217275 4626 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_85c5c1d72c47496b94eb0f8c080ead12 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:29:06.218604 4621 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_68574d2015dd47bbb92137d783bad572 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:29:06.221273 4625 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1fbfdf99ff4e4165bfcf02ceb0247c87 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:29:06.228621 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9f07e8a37404636bda765146b20fc60 consensus state has no leader
I20260927 23:29:06.230484 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f07e8a37404636bda765146b20fc60" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:06.231101 5115 raft_consensus.cc:3037] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:06.237910 5115 raft_consensus.cc:2454] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 1.
I20260927 23:29:06.238919 4769 leader_election.cc:302] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:06.239655 5790 raft_consensus.cc:2781] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:06.240048 5790 raft_consensus.cc:686] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: Becoming Leader. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Running, Role: LEADER
I20260927 23:29:06.240871 5790 consensus_queue.cc:237] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.252141 5783 leader_election.cc:288] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Requested vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:06.253954 5229 ts_tablet_manager.cc:1428] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.058s user 0.004s sys 0.000s
I20260927 23:29:06.257014 5229 tablet_bootstrap.cc:492] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:06.212450 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9f07e8a37404636bda765146b20fc60" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:06.273591 5437 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:06.274343 4972 raft_consensus.cc:3037] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:06.274052 5437 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:06.275560 5229 tablet_bootstrap.cc:654] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:06.275897 5437 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:06.286824 4972 raft_consensus.cc:2454] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 1.
I20260927 23:29:06.239497 5580 ts_tablet_manager.cc:1428] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.103s user 0.006s sys 0.001s
I20260927 23:29:06.295311 5580 tablet_bootstrap.cc:492] T be4adc02302b44c2b20a2be202ebe68d P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
W20260927 23:29:06.307181 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:29:06.308284 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:29:06.308651 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:06.309962 5718 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:06.328441 5702 tablet_bootstrap.cc:492] T be4adc02302b44c2b20a2be202ebe68d P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:06.328904 5702 ts_tablet_manager.cc:1397] T be4adc02302b44c2b20a2be202ebe68d P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.531s user 0.012s sys 0.001s
I20260927 23:29:06.332242 5702 raft_consensus.cc:348] T be4adc02302b44c2b20a2be202ebe68d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.332944 5702 raft_consensus.cc:374] T be4adc02302b44c2b20a2be202ebe68d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:06.333211 5702 raft_consensus.cc:729] T be4adc02302b44c2b20a2be202ebe68d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:06.333880 5702 consensus_queue.cc:260] T be4adc02302b44c2b20a2be202ebe68d P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.336860 5543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 143 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269" is_pre_election: true
I20260927 23:29:06.337815 5543 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f8460f73b40c41a28ec020fc04203564 in term 1.
I20260927 23:29:06.340076 4537 leader_election.cc:302] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564; no voters: 83cd22a89b7846999c0e4dfab80ebe9d
I20260927 23:29:06.341331 5781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 23:29:06.341675 5781 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 83cd22a89b7846999c0e4dfab80ebe9d)
I20260927 23:29:06.341993 5781 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 1 FOLLOWER]: Advancing to term 2
I20260927 23:29:06.357599 5229 tablet_bootstrap.cc:492] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:06.358048 5229 ts_tablet_manager.cc:1397] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.101s user 0.021s sys 0.000s
I20260927 23:29:06.358337 4623 catalog_manager.cc:5587] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571) does not have the latest schema for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]). Expected version 1 got 0
I20260927 23:29:06.361006 4623 master_service.cc:946] ReplaceTablet: received request to replace tablet 0e0aaa4d2fc8491aaf77eda6aa94616f from {username='slave'} at 127.0.0.1:35136
I20260927 23:29:06.360216 5781 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:06.380239 4560 raft_consensus.cc:1226] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 2 FOLLOWER]: Rejecting Update request from peer 83cd22a89b7846999c0e4dfab80ebe9d for earlier term 1. Current term is 2. Ops: []
I20260927 23:29:06.390472 5793 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 }, Status: INVALID_TERM, Last received: 1.145, Next index: 146, Last known committed idx: 143, Time since last communication: 0.001s
I20260927 23:29:06.391628 5781 leader_election.cc:288] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 election: Requested vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:29:06.391736 5788 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 1 LEADER]: Stepping down as leader of term 1
I20260927 23:29:06.392096 5788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 83cd22a89b7846999c0e4dfab80ebe9d, State: Running, Role: LEADER
I20260927 23:29:06.392681 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 145 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d"
I20260927 23:29:06.392845 5788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 145, Committed index: 145, Last appended: 1.146, Last appended by leader: 146, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:06.393805 5788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 1 FOLLOWER]: Advancing to term 2
I20260927 23:29:06.360121 5229 raft_consensus.cc:348] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.396847 5229 raft_consensus.cc:374] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:06.397089 5229 raft_consensus.cc:729] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:06.401057 5543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8460f73b40c41a28ec020fc04203564" candidate_term: 2 candidate_status { last_received { term: 1 index: 145 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269"
I20260927 23:29:06.401614 5543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 1 FOLLOWER]: Advancing to term 2
I20260927 23:29:06.413257 5229 consensus_queue.cc:260] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.414470 5118 raft_consensus.cc:1261] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Refusing update from remote peer bad92a10de2c4459827cdbb18854d76d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:29:06.415799 5229 ts_tablet_manager.cc:1428] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.057s user 0.006s sys 0.001s
I20260927 23:29:06.416662 5700 consensus_queue.cc:1035] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:06.422832 4830 raft_consensus.cc:1261] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Refusing update from remote peer bad92a10de2c4459827cdbb18854d76d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 23:29:06.423152 4831 raft_consensus.cc:3037] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:06.424945 5681 consensus_queue.cc:1035] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:06.427489 5543 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8460f73b40c41a28ec020fc04203564 in term 2.
I20260927 23:29:06.429605 4537 leader_election.cc:302] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564; no voters:
I20260927 23:29:06.430379 5781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 23:29:06.437703 5580 tablet_bootstrap.cc:654] T be4adc02302b44c2b20a2be202ebe68d P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:06.440879 5229 tablet_bootstrap.cc:492] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:06.442323 5781 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [term 2 LEADER]: Becoming Leader. State: Replica: f8460f73b40c41a28ec020fc04203564, State: Running, Role: LEADER
I20260927 23:29:06.443383 5781 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 143, Committed index: 143, Last appended: 1.145, Last appended by leader: 144, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:06.449606 5781 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8460f73b40c41a28ec020fc04203564. Latest consensus state: current_term: 2 leader_uuid: "f8460f73b40c41a28ec020fc04203564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:06.451282 5781 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: This master's current role is: LEADER
I20260927 23:29:06.453122 4640 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f8460f73b40c41a28ec020fc04203564 for term 2 because replica has last-logged OpId of term: 1 index: 146, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 145.
I20260927 23:29:06.466542 5229 tablet_bootstrap.cc:654] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:06.469921 5804 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:06.469658 5810 tablet.cc:1722] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:29:06.470333 5804 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:06.472077 5804 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:06.478732 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:29:06.479803 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:29:06.480130 5051 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:06.481138 5534 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:06.489689 5702 ts_tablet_manager.cc:1428] T be4adc02302b44c2b20a2be202ebe68d P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.161s user 0.005s sys 0.000s
I20260927 23:29:06.490792 5702 tablet_bootstrap.cc:492] T 935c9a2e4d6c416db02443d2aaa57752 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:06.501489 5804 raft_consensus.cc:482] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:06.501925 5804 raft_consensus.cc:504] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.502533 5229 tablet_bootstrap.cc:492] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:06.503139 5229 ts_tablet_manager.cc:1397] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.062s user 0.007s sys 0.002s
I20260927 23:29:06.504398 5804 leader_election.cc:288] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:06.505849 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "340a88a1702d48be9a8373fb3c6d540c" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:06.507344 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "340a88a1702d48be9a8373fb3c6d540c" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:06.507930 4972 raft_consensus.cc:2454] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:06.509294 5054 leader_election.cc:302] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:06.510264 5534 raft_consensus.cc:2781] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:06.510552 5534 raft_consensus.cc:482] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:06.510828 5534 raft_consensus.cc:3037] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:06.505218 5229 raft_consensus.cc:348] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.514210 5229 raft_consensus.cc:374] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:06.514472 5229 raft_consensus.cc:729] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
W20260927 23:29:06.517576 5051 leader_election.cc:341] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:06.517750 5229 consensus_queue.cc:260] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.523356 5534 raft_consensus.cc:504] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.523883 5580 tablet_bootstrap.cc:492] T be4adc02302b44c2b20a2be202ebe68d P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:06.524289 5580 ts_tablet_manager.cc:1397] T be4adc02302b44c2b20a2be202ebe68d P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.229s user 0.009s sys 0.000s
I20260927 23:29:06.526239 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "340a88a1702d48be9a8373fb3c6d540c" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:06.526772 5580 raft_consensus.cc:348] T be4adc02302b44c2b20a2be202ebe68d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.527367 5580 raft_consensus.cc:374] T be4adc02302b44c2b20a2be202ebe68d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:06.527279 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "340a88a1702d48be9a8373fb3c6d540c" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:06.527655 5580 raft_consensus.cc:729] T be4adc02302b44c2b20a2be202ebe68d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:06.527961 4972 raft_consensus.cc:3037] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:06.529573 5229 ts_tablet_manager.cc:1428] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.025s user 0.005s sys 0.000s
W20260927 23:29:06.532846 5051 leader_election.cc:341] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:06.533819 5804 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:06.534272 5804 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.536337 5804 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:06.538439 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:29:06.540400 5534 leader_election.cc:288] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:06.542704 4972 raft_consensus.cc:2454] T 340a88a1702d48be9a8373fb3c6d540c P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:06.543951 5054 leader_election.cc:302] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca
I20260927 23:29:06.546766 5534 raft_consensus.cc:2781] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:06.547279 5534 raft_consensus.cc:686] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
W20260927 23:29:06.548547 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:29:06.548952 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:04.823271 5544 rpcz_store.cc:276] 0927 23:29:02.937869 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:02.959349 (+ 21480us) service_pool.cc:227] Handling call
0927 23:29:03.159446 (+200097us) raft_consensus.cc:1457] Updating replica for 110 ops
0927 23:29:03.291338 (+131892us) raft_consensus.cc:1520] Early marking committed up to index 0
0927 23:29:03.291394 (+ 56us) raft_consensus.cc:1525] Triggering prepare for 110 ops
0927 23:29:03.335074 (+ 43680us) write_op.cc:270] Start()
0927 23:29:03.335405 (+ 331us) write_op.cc:276] Timestamp: P: 1790551731786359 usec, L: 0
0927 23:29:03.463227 (+127822us) write_op.cc:270] Start()
0927 23:29:03.463354 (+ 127us) write_op.cc:276] Timestamp: P: 1790551731858067 usec, L: 0
0927 23:29:03.464032 (+ 678us) write_op.cc:270] Start()
0927 23:29:03.465413 (+ 1381us) write_op.cc:276] Timestamp: P: 1790551731864828 usec, L: 0
0927 23:29:03.479159 (+ 13746us) write_op.cc:270] Start()
0927 23:29:03.479275 (+ 116us) write_op.cc:276] Timestamp: P: 1790551731888045 usec, L: 0
0927 23:29:03.486710 (+ 7435us) write_op.cc:270] Start()
0927 23:29:03.486811 (+ 101us) write_op.cc:276] Timestamp: P: 1790551731917121 usec, L: 0
0927 23:29:03.491265 (+ 4454us) write_op.cc:270] Start()
0927 23:29:03.491385 (+ 120us) write_op.cc:276] Timestamp: P: 1790551731994194 usec, L: 0
0927 23:29:03.491927 (+ 542us) write_op.cc:270] Start()
0927 23:29:03.495927 (+ 4000us) write_op.cc:276] Timestamp: P: 1790551732028223 usec, L: 0
0927 23:29:03.496508 (+ 581us) write_op.cc:270] Start()
0927 23:29:03.496593 (+ 85us) write_op.cc:276] Timestamp: P: 1790551732078909 usec, L: 0
0927 23:29:03.497060 (+ 467us) write_op.cc:270] Start()
0927 23:29:03.497138 (+ 78us) write_op.cc:276] Timestamp: P: 1790551732166254 usec, L: 0
0927 23:29:03.497546 (+ 408us) write_op.cc:270] Start()
0927 23:29:03.497621 (+ 75us) write_op.cc:276] Timestamp: P: 1790551732226761 usec, L: 0
0927 23:29:03.498014 (+ 393us) write_op.cc:270] Start()
0927 23:29:03.498085 (+ 71us) write_op.cc:276] Timestamp: P: 1790551732258577 usec, L: 0
0927 23:29:03.498485 (+ 400us) write_op.cc:270] Start()
0927 23:29:03.498589 (+ 104us) write_op.cc:276] Timestamp: P: 1790551732317233 usec, L: 0
0927 23:29:03.498993 (+ 404us) write_op.cc:270] Start()
0927 23:29:03.499071 (+ 78us) write_op.cc:276] Timestamp: P: 1790551732413448 usec, L: 0
0927 23:29:03.499442 (+ 371us) write_op.cc:270] Start()
0927 23:29:03.499505 (+ 63us) write_op.cc:276] Timestamp: P: 1790551732472540 usec, L: 0
0927 23:29:03.499899 (+ 394us) write_op.cc:270] Start()
0927 23:29:03.499977 (+ 78us) write_op.cc:276] Timestamp: P: 1790551732539443 usec, L: 0
0927 23:29:03.577736 (+ 77759us) write_op.cc:270] Start()
0927 23:29:03.577858 (+ 122us) write_op.cc:276] Timestamp: P: 1790551732608383 usec, L: 0
0927 23:29:03.637202 (+ 59344us) write_op.cc:270] Start()
0927 23:29:03.637320 (+ 118us) write_op.cc:276] Timestamp: P: 1790551732741800 usec, L: 0
0927 23:29:03.638098 (+ 778us) write_op.cc:270] Start()
0927 23:29:03.638186 (+ 88us) write_op.cc:276] Timestamp: P: 1790551732811439 usec, L: 0
0927 23:29:03.638673 (+ 487us) write_op.cc:270] Start()
0927 23:29:03.638754 (+ 81us) write_op.cc:276] Timestamp: P: 1790551732889461 usec, L: 0
0927 23:29:03.639187 (+ 433us) write_op.cc:270] Start()
0927 23:29:03.639270 (+ 83us) write_op.cc:276] Timestamp: P: 1790551732929101 usec, L: 0
0927 23:29:03.639681 (+ 411us) write_op.cc:270] Start()
0927 23:29:03.639759 (+ 78us) write_op.cc:276] Timestamp: P: 1790551733010632 usec, L: 0
0927 23:29:03.640174 (+ 415us) write_op.cc:270] Start()
0927 23:29:03.640242 (+ 68us) write_op.cc:276] Timestamp: P: 1790551733145757 usec, L: 0
0927 23:29:03.673132 (+ 32890us) write_op.cc:270] Start()
0927 23:29:03.673255 (+ 123us) write_op.cc:276] Timestamp: P: 1790551733180659 usec, L: 0
0927 23:29:03.673773 (+ 518us) write_op.cc:270] Start()
0927 23:29:03.673896 (+ 123us) write_op.cc:276] Timestamp: P: 1790551733279214 usec, L: 0
0927 23:29:03.674502 (+ 606us) write_op.cc:270] Start()
0927 23:29:03.674578 (+ 76us) write_op.cc:276] Timestamp: P: 1790551733410059 usec, L: 0
0927 23:29:03.675004 (+ 426us) write_op.cc:270] Start()
0927 23:29:03.675118 (+ 114us) write_op.cc:276] Timestamp: P: 1790551733488617 usec, L: 0
0927 23:29:03.675580 (+ 462us) write_op.cc:270] Start()
0927 23:29:03.675656 (+ 76us) write_op.cc:276] Timestamp: P: 1790551733532896 usec, L: 0
0927 23:29:03.676058 (+ 402us) write_op.cc:270] Start()
0927 23:29:03.676128 (+ 70us) write_op.cc:276] Timestamp: P: 1790551733662993 usec, L: 0
0927 23:29:03.696937 (+ 20809us) write_op.cc:270] Start()
0927 23:29:03.697050 (+ 113us) write_op.cc:276] Timestamp: P: 1790551733776370 usec, L: 0
0927 23:29:03.697604 (+ 554us) write_op.cc:270] Start()
0927 23:29:03.697683 (+ 79us) write_op.cc:276] Timestamp: P: 1790551733918795 usec, L: 0
0927 23:29:03.698109 (+ 426us) write_op.cc:270] Start()
0927 23:29:03.698190 (+ 81us) write_op.cc:276] Timestamp: P: 1790551733970834 usec, L: 0
0927 23:29:03.698644 (+ 454us) write_op.cc:270] Start()
0927 23:29:03.698724 (+ 80us) write_op.cc:276] Timestamp: P: 1790551734110161 usec, L: 0
0927 23:29:03.699210 (+ 486us) write_op.cc:270] Start()
0927 23:29:03.699285 (+ 75us) write_op.cc:276] Timestamp: P: 1790551734172699 usec, L: 0
0927 23:29:03.699788 (+ 503us) write_op.cc:270] Start()
0927 23:29:03.699867 (+ 79us) write_op.cc:276] Timestamp: P: 1790551734199659 usec, L: 0
0927 23:29:03.700274 (+ 407us) write_op.cc:270] Start()
0927 23:29:03.733155 (+ 32881us) write_op.cc:276] Timestamp: P: 1790551734295950 usec, L: 0
0927 23:29:03.735294 (+ 2139us) write_op.cc:270] Start()
0927 23:29:03.735394 (+ 100us) write_op.cc:276] Timestamp: P: 1790551734324499 usec, L: 0
0927 23:29:03.735923 (+ 529us) write_op.cc:270] Start()
0927 23:29:03.736009 (+ 86us) write_op.cc:276] Timestamp: P: 1790551734470992 usec, L: 0
0927 23:29:03.736469 (+ 460us) write_op.cc:270] Start()
0927 23:29:03.736555 (+ 86us) write_op.cc:276] Timestamp: P: 1790551734588387 usec, L: 0
0927 23:29:03.737010 (+ 455us) write_op.cc:270] Start()
0927 23:29:03.737089 (+ 79us) write_op.cc:276] Timestamp: P: 1790551734637856 usec, L: 0
0927 23:29:03.737537 (+ 448us) write_op.cc:270] Start()
0927 23:29:03.737627 (+ 90us) write_op.cc:276] Timestamp: P: 1790551734728152 usec, L: 0
0927 23:29:03.738064 (+ 437us) write_op.cc:270] Start()
0927 23:29:03.738137 (+ 73us) write_op.cc:276] Timestamp: P: 1790551734828081 usec, L: 0
0927 23:29:03.738564 (+ 427us) write_op.cc:270] Start()
0927 23:29:03.738631 (+ 67us) write_op.cc:276] Timestamp: P: 1790551734919815 usec, L: 0
0927 23:29:03.739047 (+ 416us) write_op.cc:270] Start()
0927 23:29:03.739124 (+ 77us) write_op.cc:276] Timestamp: P: 1790551734954298 usec, L: 0
0927 23:29:03.739570 (+ 446us) write_op.cc:270] Start()
0927 23:29:03.739653 (+ 83us) write_op.cc:276] Timestamp: P: 1790551735067080 usec, L: 0
0927 23:29:03.740073 (+ 420us) write_op.cc:270] Start()
0927 23:29:03.774343 (+ 34270us) write_op.cc:276] Timestamp: P: 1790551735095098 usec, L: 0
0927 23:29:03.775017 (+ 674us) write_op.cc:270] Start()
0927 23:29:03.775103 (+ 86us) write_op.cc:276] Timestamp: P: 1790551735186684 usec, L: 0
0927 23:29:03.775570 (+ 467us) write_op.cc:270] Start()
0927 23:29:03.775648 (+ 78us) write_op.cc:276] Timestamp: P: 1790551735229997 usec, L: 0
0927 23:29:03.776089 (+ 441us) write_op.cc:270] Start()
0927 23:29:03.776240 (+ 151us) write_op.cc:276] Timestamp: P: 1790551735266838 usec, L: 0
0927 23:29:03.776847 (+ 607us) write_op.cc:270] Start()
0927 23:29:03.776921 (+ 74us) write_op.cc:276] Timestamp: P: 1790551735332247 usec, L: 0
0927 23:29:03.777348 (+ 427us) write_op.cc:270] Start()
0927 23:29:03.777414 (+ 66us) write_op.cc:276] Timestamp: P: 1790551735443737 usec, L: 0
0927 23:29:03.777844 (+ 430us) write_op.cc:270] Start()
0927 23:29:03.777924 (+ 80us) write_op.cc:276] Timestamp: P: 1790551735486913 usec, L: 0
0927 23:29:03.778356 (+ 432us) write_op.cc:270] Start()
0927 23:29:03.778430 (+ 74us) write_op.cc:276] Timestamp: P: 1790551735564801 usec, L: 0
0927 23:29:03.819408 (+ 40978us) write_op.cc:270] Start()
0927 23:29:03.819531 (+ 123us) write_op.cc:276] Timestamp: P: 1790551735607737 usec, L: 0
0927 23:29:03.838467 (+ 18936us) write_op.cc:270] Start()
0927 23:29:03.838577 (+ 110us) write_op.cc:276] Timestamp: P: 1790551735748861 usec, L: 0
0927 23:29:03.839139 (+ 562us) write_op.cc:270] Start()
0927 23:29:03.839232 (+ 93us) write_op.cc:276] Timestamp: P: 1790551735769514 usec, L: 0
0927 23:29:03.839731 (+ 499us) write_op.cc:270] Start()
0927 23:29:03.839816 (+ 85us) write_op.cc:276] Timestamp: P: 1790551735991865 usec, L: 0
0927 23:29:03.840255 (+ 439us) write_op.cc:270] Start()
0927 23:29:03.840366 (+ 111us) write_op.cc:276] Timestamp: P: 1790551736111957 usec, L: 0
0927 23:29:03.840814 (+ 448us) write_op.cc:270] Start()
0927 23:29:03.840895 (+ 81us) write_op.cc:276] Timestamp: P: 1790551736174167 usec, L: 0
0927 23:29:03.862988 (+ 22093us) write_op.cc:270] Start()
0927 23:29:03.863098 (+ 110us) write_op.cc:276] Timestamp: P: 1790551736246264 usec, L: 0
0927 23:29:03.863732 (+ 634us) write_op.cc:270] Start()
0927 23:29:03.863908 (+ 176us) write_op.cc:276] Timestamp: P: 1790551736268373 usec, L: 0
0927 23:29:03.864441 (+ 533us) write_op.cc:270] Start()
0927 23:29:03.864528 (+ 87us) write_op.cc:276] Timestamp: P: 1790551736555860 usec, L: 0
0927 23:29:03.865004 (+ 476us) write_op.cc:270] Start()
0927 23:29:03.865080 (+ 76us) write_op.cc:276] Timestamp: P: 1790551736673158 usec, L: 0
0927 23:29:03.865498 (+ 418us) write_op.cc:270] Start()
0927 23:29:03.865574 (+ 76us) write_op.cc:276] Timestamp: P: 1790551736763872 usec, L: 0
0927 23:29:03.865989 (+ 415us) write_op.cc:270] Start()
0927 23:29:03.866064 (+ 75us) write_op.cc:276] Timestamp: P: 1790551736788945 usec, L: 0
0927 23:29:03.866707 (+ 643us) write_op.cc:270] Start()
0927 23:29:03.866779 (+ 72us) write_op.cc:276] Timestamp: P: 1790551736829350 usec, L: 0
0927 23:29:03.867184 (+ 405us) write_op.cc:270] Start()
0927 23:29:03.867261 (+ 77us) write_op.cc:276] Timestamp: P: 1790551736944844 usec, L: 0
0927 23:29:03.867688 (+ 427us) write_op.cc:270] Start()
0927 23:29:03.867762 (+ 74us) write_op.cc:276] Timestamp: P: 1790551737135565 usec, L: 0
0927 23:29:03.868177 (+ 415us) write_op.cc:270] Start()
0927 23:29:03.868255 (+ 78us) write_op.cc:276] Timestamp: P: 1790551737249509 usec, L: 0
0927 23:29:03.909750 (+ 41495us) write_op.cc:270] Start()
0927 23:29:03.909851 (+ 101us) write_op.cc:276] Timestamp: P: 1790551737313084 usec, L: 0
0927 23:29:03.910373 (+ 522us) write_op.cc:270] Start()
0927 23:29:03.910460 (+ 87us) write_op.cc:276] Timestamp: P: 1790551737438779 usec, L: 0
0927 23:29:03.910920 (+ 460us) write_op.cc:270] Start()
0927 23:29:03.910996 (+ 76us) write_op.cc:276] Timestamp: P: 1790551737478486 usec, L: 0
0927 23:29:03.911396 (+ 400us) write_op.cc:270] Start()
0927 23:29:03.911471 (+ 75us) write_op.cc:276] Timestamp: P: 1790551737539156 usec, L: 0
0927 23:29:03.911910 (+ 439us) write_op.cc:270] Start()
0927 23:29:03.911983 (+ 73us) write_op.cc:276] Timestamp: P: 1790551737563247 usec, L: 0
0927 23:29:03.968523 (+ 56540us) write_op.cc:270] Start()
0927 23:29:03.968642 (+ 119us) write_op.cc:276] Timestamp: P: 1790551737619966 usec, L: 0
0927 23:29:03.969330 (+ 688us) write_op.cc:270] Start()
0927 23:29:03.969414 (+ 84us) write_op.cc:276] Timestamp: P: 1790551737847013 usec, L: 0
0927 23:29:03.969882 (+ 468us) write_op.cc:270] Start()
0927 23:29:03.969979 (+ 97us) write_op.cc:276] Timestamp: P: 1790551737987551 usec, L: 0
0927 23:29:03.970452 (+ 473us) write_op.cc:270] Start()
0927 23:29:03.970530 (+ 78us) write_op.cc:276] Timestamp: P: 1790551738020487 usec, L: 0
0927 23:29:03.970925 (+ 395us) write_op.cc:270] Start()
0927 23:29:03.970995 (+ 70us) write_op.cc:276] Timestamp: P: 1790551738192668 usec, L: 0
0927 23:29:03.971424 (+ 429us) write_op.cc:270] Start()
0927 23:29:03.971497 (+ 73us) write_op.cc:276] Timestamp: P: 1790551738274862 usec, L: 0
0927 23:29:03.971905 (+ 408us) write_op.cc:270] Start()
0927 23:29:03.971978 (+ 73us) write_op.cc:276] Timestamp: P: 1790551738335458 usec, L: 0
0927 23:29:04.015649 (+ 43671us) write_op.cc:270] Start()
0927 23:29:04.015773 (+ 124us) write_op.cc:276] Timestamp: P: 1790551738371750 usec, L: 0
0927 23:29:04.016323 (+ 550us) write_op.cc:270] Start()
0927 23:29:04.016406 (+ 83us) write_op.cc:276] Timestamp: P: 1790551738520204 usec, L: 0
0927 23:29:04.016835 (+ 429us) write_op.cc:270] Start()
0927 23:29:04.016907 (+ 72us) write_op.cc:276] Timestamp: P: 1790551738661191 usec, L: 0
0927 23:29:04.017549 (+ 642us) write_op.cc:270] Start()
0927 23:29:04.017635 (+ 86us) write_op.cc:276] Timestamp: P: 1790551738734310 usec, L: 0
0927 23:29:04.018081 (+ 446us) write_op.cc:270] Start()
0927 23:29:04.018165 (+ 84us) write_op.cc:276] Timestamp: P: 1790551738905616 usec, L: 0
0927 23:29:04.018578 (+ 413us) write_op.cc:270] Start()
0927 23:29:04.018659 (+ 81us) write_op.cc:276] Timestamp: P: 1790551738966655 usec, L: 0
0927 23:29:04.019079 (+ 420us) write_op.cc:270] Start()
0927 23:29:04.019154 (+ 75us) write_op.cc:276] Timestamp: P: 1790551739051675 usec, L: 0
0927 23:29:04.019561 (+ 407us) write_op.cc:270] Start()
0927 23:29:04.019630 (+ 69us) write_op.cc:276] Timestamp: P: 1790551739301428 usec, L: 0
0927 23:29:04.020076 (+ 446us) write_op.cc:270] Start()
0927 23:29:04.020150 (+ 74us) write_op.cc:276] Timestamp: P: 1790551739356853 usec, L: 0
0927 23:29:04.028706 (+ 8556us) write_op.cc:270] Start()
0927 23:29:04.028820 (+ 114us) write_op.cc:276] Timestamp: P: 1790551739428035 usec, L: 0
0927 23:29:04.029300 (+ 480us) write_op.cc:270] Start()
0927 23:29:04.029379 (+ 79us) write_op.cc:276] Timestamp: P: 1790551739681396 usec, L: 0
0927 23:29:04.056693 (+ 27314us) write_op.cc:270] Start()
0927 23:29:04.056831 (+ 138us) write_op.cc:276] Timestamp: P: 1790551739911856 usec, L: 0
0927 23:29:04.057420 (+ 589us) write_op.cc:270] Start()
0927 23:29:04.057499 (+ 79us) write_op.cc:276] Timestamp: P: 1790551739958886 usec, L: 0
0927 23:29:04.057960 (+ 461us) write_op.cc:270] Start()
0927 23:29:04.058045 (+ 85us) write_op.cc:276] Timestamp: P: 1790551740018001 usec, L: 0
0927 23:29:04.058484 (+ 439us) write_op.cc:270] Start()
0927 23:29:04.058572 (+ 88us) write_op.cc:276] Timestamp: P: 1790551740212845 usec, L: 0
0927 23:29:04.125251 (+ 66679us) write_op.cc:270] Start()
0927 23:29:04.125358 (+ 107us) write_op.cc:276] Timestamp: P: 1790551740265212 usec, L: 0
0927 23:29:04.125914 (+ 556us) write_op.cc:270] Start()
0927 23:29:04.126000 (+ 86us) write_op.cc:276] Timestamp: P: 1790551740410680 usec, L: 0
0927 23:29:04.126462 (+ 462us) write_op.cc:270] Start()
0927 23:29:04.126645 (+ 183us) write_op.cc:276] Timestamp: P: 1790551740469415 usec, L: 0
0927 23:29:04.127214 (+ 569us) write_op.cc:270] Start()
0927 23:29:04.127297 (+ 83us) write_op.cc:276] Timestamp: P: 1790551741022931 usec, L: 0
0927 23:29:04.127744 (+ 447us) write_op.cc:270] Start()
0927 23:29:04.127822 (+ 78us) write_op.cc:276] Timestamp: P: 1790551741034184 usec, L: 0
0927 23:29:04.128224 (+ 402us) write_op.cc:270] Start()
0927 23:29:04.171251 (+ 43027us) write_op.cc:276] Timestamp: P: 1790551741090863 usec, L: 0
0927 23:29:04.171871 (+ 620us) write_op.cc:270] Start()
0927 23:29:04.171977 (+ 106us) write_op.cc:276] Timestamp: P: 1790551741283408 usec, L: 0
0927 23:29:04.172465 (+ 488us) write_op.cc:270] Start()
0927 23:29:04.172542 (+ 77us) write_op.cc:276] Timestamp: P: 1790551741383792 usec, L: 0
0927 23:29:04.172988 (+ 446us) write_op.cc:270] Start()
0927 23:29:04.173070 (+ 82us) write_op.cc:276] Timestamp: P: 1790551741436752 usec, L: 0
0927 23:29:04.173510 (+ 440us) write_op.cc:270] Start()
0927 23:29:04.173586 (+ 76us) write_op.cc:276] Timestamp: P: 1790551741575817 usec, L: 0
0927 23:29:04.174203 (+ 617us) write_op.cc:270] Start()
0927 23:29:04.174283 (+ 80us) write_op.cc:276] Timestamp: P: 1790551741813051 usec, L: 0
0927 23:29:04.174700 (+ 417us) write_op.cc:270] Start()
0927 23:29:04.204363 (+ 29663us) write_op.cc:276] Timestamp: P: 1790551741983930 usec, L: 0
0927 23:29:04.205038 (+ 675us) write_op.cc:270] Start()
0927 23:29:04.205128 (+ 90us) write_op.cc:276] Timestamp: P: 1790551742023584 usec, L: 0
0927 23:29:04.205599 (+ 471us) write_op.cc:270] Start()
0927 23:29:04.205687 (+ 88us) write_op.cc:276] Timestamp: P: 1790551742155559 usec, L: 0
0927 23:29:04.206181 (+ 494us) write_op.cc:270] Start()
0927 23:29:04.206258 (+ 77us) write_op.cc:276] Timestamp: P: 1790551742176671 usec, L: 0
0927 23:29:04.408934 (+202676us) log.cc:842] Serialized 60338 byte log entry
0927 23:29:04.445155 (+ 36221us) raft_consensus.cc:1631] Marking committed up to 114
0927 23:29:04.630277 (+185122us) raft_consensus.cc:1683] Filling consensus response to leader.
0927 23:29:04.647395 (+ 17118us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400010754. stack: 00007ffb50419033 00007ffb50418f30 00007ffb4ff9a534 00000000004db2b4 00000000004db249 00007ffb57325af0 00007ffb573a8f2a 00007ffb593a030c 00007ffb593a872c 00007ffb593a86d8 00007ffb593a8658 00007ffb593a8613 00007ffb593a78bc 00007ffb577d90e6 00007ffb577d9078 00007ffb573bf2fa
0927 23:29:04.774934 (+127539us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0927 23:29:04.822273 (+ 47339us) raft_consensus.cc:1671] finished
0927 23:29:04.822304 (+ 31us) raft_consensus.cc:1677] UpdateReplicas() finished
0927 23:29:04.822711 (+ 407us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:03.462452 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:03.464964 (+ 2512us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:03.465003 (+ 39us) write_op.cc:435] Acquired schema lock
0927 23:29:03.465027 (+ 24us) tablet.cc:660] Decoding operations
0927 23:29:03.472851 (+ 7824us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:03.473013 (+ 162us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:03.473052 (+ 39us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:29:03.482381 (+ 9329us) tablet.cc:699] Row locks acquired
0927 23:29:03.482416 (+ 35us) write_op.cc:260] PREPARE: finished
0927 23:29:04.445385 (+962969us) op_driver.cc:444] REPLICATION: finished
0927 23:29:04.449149 (+ 3764us) write_op.cc:301] APPLY: starting
0927 23:29:04.449226 (+ 77us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:04.449430 (+ 204us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:04.449449 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:04.449850 (+ 401us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:04.450140 (+ 290us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:04.450282 (+ 142us) write_op.cc:312] APPLY: finished
0927 23:29:04.450618 (+ 336us) log.cc:842] Serialized 28 byte log entry
0927 23:29:04.450964 (+ 346us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.468711 (+ 17747us) write_op.cc:454] Released schema lock
0927 23:29:04.468745 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:04.468956 (+ 211us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.468981 (+ 25us) write_op.cc:454] Released schema lock
Related trace 'op':
0927 23:29:03.482734 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:03.487185 (+ 4451us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:03.487219 (+ 34us) write_op.cc:435] Acquired schema lock
0927 23:29:03.487236 (+ 17us) tablet.cc:660] Decoding operations
0927 23:29:03.488175 (+ 939us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:03.488279 (+ 104us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:03.488330 (+ 51us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:29:03.488573 (+ 243us) tablet.cc:699] Row locks acquired
0927 23:29:03.488586 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:29:04.460552 (+971966us) op_driver.cc:444] REPLICATION: finished
0927 23:29:04.462236 (+ 1684us) write_op.cc:301] APPLY: starting
0927 23:29:04.462294 (+ 58us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:04.462391 (+ 97us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:04.462410 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:04.462901 (+ 491us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:04.469251 (+ 6350us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:04.469388 (+ 137us) write_op.cc:312] APPLY: finished
0927 23:29:04.469580 (+ 192us) log.cc:842] Serialized 28 byte log entry
0927 23:29:04.469883 (+ 303us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.472487 (+ 2604us) write_op.cc:454] Released schema lock
0927 23:29:04.472517 (+ 30us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:04.472637 (+ 120us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.472656 (+ 19us) write_op.cc:454] Released schema lock
Related trace 'op':
0927 23:29:03.488853 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:03.489277 (+ 424us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:03.489303 (+ 26us) write_op.cc:435] Acquired schema lock
0927 23:29:03.489319 (+ 16us) tablet.cc:660] Decoding operations
0927 23:29:03.489690 (+ 371us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:03.489750 (+ 60us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:03.489774 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:29:04.451073 (+961299us) lock_manager.cc:599] Waited 960956us for lock on <redacted> (key hash 7889546944618713878)
0927 23:29:04.451114 (+ 41us) tablet.cc:699] Row locks acquired
0927 23:29:04.451126 (+ 12us) write_op.cc:260] PREPARE: finished
0927 23:29:04.464410 (+ 13284us) op_driver.cc:444] REPLICATION: finished
0927 23:29:04.467085 (+ 2675us) write_op.cc:301] APPLY: starting
0927 23:29:04.467137 (+ 52us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:04.467230 (+ 93us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:04.467249 (+ 19us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:04.467471 (+ 222us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:04.467671 (+ 200us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:29:04.467715 (+ 44us) write_op.cc:312] APPLY: finished
0927 23:29:04.467924 (+ 209us) log.cc:842] Serialized 22 byte log entry
0927 23:29:04.468047 (+ 123us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.468098 (+ 51us) write_op.cc:454] Released schema lock
0927 23:29:04.468145 (+ 47us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:04.468400 (+ 255us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.468426 (+ 26us) write_op.cc:454] Released schema lock
Related trace 'op':
0927 23:29:04.451393 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:04.451823 (+ 430us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:04.451847 (+ 24us) write_op.cc:435] Acquired schema lock
0927 23:29:04.451862 (+ 15us) tablet.cc:660] Decoding operations
0927 23:29:04.452116 (+ 254us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:04.452171 (+ 55us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:04.452194 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:29:04.452395 (+ 201us) tablet.cc:699] Row locks acquired
0927 23:29:04.452408 (+ 13us) write_op.cc:260] PREPARE: finished
0927 23:29:04.506729 (+ 54321us) op_driver.cc:444] REPLICATION: finished
0927 23:29:04.510434 (+ 3705us) write_op.cc:301] APPLY: starting
0927 23:29:04.510494 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:04.510590 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:04.510611 (+ 21us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:04.510849 (+ 238us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:04.510975 (+ 126us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:04.511082 (+ 107us) write_op.cc:312] APPLY: finished
0927 23:29:04.511264 (+ 182us) log.cc:842] Serialized 22 byte log entry
0927 23:29:04.511528 (+ 264us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.511583 (+ 55us) write_op.cc:454] Released schema lock
0927 23:29:04.511614 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:04.511750 (+ 136us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.511772 (+ 22us) write_op.cc:454] Released schema lock
Related trace 'op':
0927 23:29:04.452629 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:04.452995 (+ 366us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:04.453019 (+ 24us) write_op.cc:435] Acquired schema lock
0927 23:29:04.453034 (+ 15us) tablet.cc:660] Decoding operations
0927 23:29:04.453290 (+ 256us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:04.453339 (+ 49us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:04.453363 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:29:04.469970 (+ 16607us) tablet.cc:699] Row locks acquired
0927 23:29:04.469989 (+ 19us) write_op.cc:260] PREPARE: finished
0927 23:29:04.506948 (+ 36959us) op_driver.cc:444] REPLICATION: finished
0927 23:29:04.512044 (+ 5096us) write_op.cc:301] APPLY: starting
0927 23:29:04.512095 (+ 51us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:04.512184 (+ 89us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:04.512200 (+ 16us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:04.560410 (+ 48210us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:04.560555 (+ 145us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:29:04.735706 (+175151us) write_op.cc:312] APPLY: finished
0927 23:29:04.735869 (+ 163us) log.cc:842] Serialized 22 byte log entry
0927 23:29:04.736228 (+ 359us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.736416 (+ 188us) write_op.cc:454] Released schema lock
0927 23:29:04.736444 (+ 28us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:04.736568 (+ 124us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.736593 (+ 25us) write_op.cc:454] Released schema lock
Related trace 'op':
0927 23:29:04.470232 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:04.470608 (+ 376us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:04.470631 (+ 23us) write_op.cc:435] Acquired schema lock
0927 23:29:04.470642 (+ 11us) tablet.cc:660] Decoding operations
0927 23:29:04.470934 (+ 292us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:04.470993 (+ 59us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:04.471020 (+ 27us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:29:04.471235 (+ 215us) tablet.cc:699] Row locks acquired
0927 23:29:04.471251 (+ 16us) write_op.cc:260] PREPARE: finished
0927 23:29:04.507126 (+ 35875us) op_driver.cc:444] REPLICATION: finished
0927 23:29:04.518468 (+ 11342us) write_op.cc:301] APPLY: starting
0927 23:29:04.518528 (+ 60us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:04.518626 (+ 98us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:04.518644 (+ 18us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:04.570697 (+ 52053us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:04.570857 (+ 160us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:04.714510 (+143653us) write_op.cc:312] APPLY: finished
0927 23:29:04.714729 (+ 219us) log.cc:842] Serialized 28 byte log entry
0927 23:29:04.714868 (+ 139us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.749200 (+ 34332us) write_op.cc:454] Released schema lock
0927 23:29:04.749241 (+ 41us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:04.749401 (+ 160us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:04.749425 (+ 24us) write_op.cc:454] Released schema lock
Related trace 'op':
0927 23:29:04.471485 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:04.471886 (+ 401us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:04.471911 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:29:04.471926 (+ 15us) tablet.cc:660] Decoding operations
0927 23:29:04.472177 (+ 251us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:04.472227 (+ 50us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:04.472250 (+ 23us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:2
I20260927 23:29:06.561144 5804 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:06.571895 5702 tablet_bootstrap.cc:654] T 935c9a2e4d6c416db02443d2aaa57752 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:06.575021 5776 tablet.cc:1722] T 1572bbcfebce4d218b1a0f38ecab0384 P bad92a10de2c4459827cdbb18854d76d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:29:06.575062 5580 consensus_queue.cc:260] T be4adc02302b44c2b20a2be202ebe68d P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.585376 4688 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 23:29:06.548170 5534 consensus_queue.cc:237] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.600045 4628 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]
I20260927 23:29:06.600596 4622 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d] alter complete (version 1)
W20260927 23:29:06.601145 4622 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.2.57.3:57013 (request call id 103) took 793 ms (client timeout 1000 ms). Trace:
W20260927 23:29:06.601446 4622 rpcz_store.cc:269] 0927 23:29:05.807848 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:06.082312 (+274464us) service_pool.cc:227] Handling call
0927 23:29:06.601094 (+518782us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:06.130561 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:06.130993 (+ 432us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:06.131019 (+ 26us) write_op.cc:435] Acquired schema lock
0927 23:29:06.131033 (+ 14us) tablet.cc:660] Decoding operations
0927 23:29:06.131327 (+ 294us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:06.131364 (+ 37us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:06.131388 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0927 23:29:06.131554 (+ 166us) tablet.cc:699] Row locks acquired
0927 23:29:06.131570 (+ 16us) write_op.cc:260] PREPARE: finished
0927 23:29:06.139857 (+ 8287us) write_op.cc:270] Start()
0927 23:29:06.139971 (+ 114us) write_op.cc:276] Timestamp: P: 1790551746139725 usec, L: 0
0927 23:29:06.139989 (+ 18us) op_driver.cc:331] REPLICATION: starting
0927 23:29:06.148245 (+ 8256us) log.cc:842] Serialized 356 byte log entry
0927 23:29:06.324014 (+175769us) op_driver.cc:444] REPLICATION: finished
0927 23:29:06.324941 (+ 927us) write_op.cc:301] APPLY: starting
0927 23:29:06.324991 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:06.325076 (+ 85us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:06.325157 (+ 81us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:06.325330 (+ 173us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:06.325443 (+ 113us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0927 23:29:06.325480 (+ 37us) write_op.cc:312] APPLY: finished
0927 23:29:06.325666 (+ 186us) log.cc:842] Serialized 23 byte log entry
0927 23:29:06.325822 (+ 156us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:06.325878 (+ 56us) write_op.cc:454] Released schema lock
0927 23:29:06.325904 (+ 26us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:06.326062 (+ 158us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:06.326083 (+ 21us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":782,"apply.run_cpu_time_us":1300,"apply.run_wall_time_us":1300,"num_ops":1,"prepare.queue_time_us":16812,"prepare.run_cpu_time_us":2255,"prepare.run_wall_time_us":19689,"replication_time_us":183946}]]}
I20260927 23:29:06.612530 4622 master_service.cc:946] ReplaceTablet: received request to replace tablet 48dcf44bb7b744b9a98626dbfb6f4e6c from {username='slave'} at 127.0.0.1:35136
W20260927 23:29:06.618628 5177 master_proxy_rpc.cc:203] Re-attempting ReplaceTablet request to leader Master (127.2.57.60:33749)
I20260927 23:29:06.644387 5814 tablet.cc:1722] T 1572bbcfebce4d218b1a0f38ecab0384 P f5ec98b2972d44838fa273eb7a10d01a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:29:06.679906 5773 mvcc.cc:205] Tried to move back new op lower bound from 7334099953275842560 to 7334099949315264512. Current Snapshot: MvccSnapshot[applied={T|T < 7334099953275842560}]
I20260927 23:29:06.693748 5810 tablet.cc:1722] T 1572bbcfebce4d218b1a0f38ecab0384 P 7e62b4926a8448b2b5f4923553328bca: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260927 23:29:06.714385 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:06.576972 5734 mvcc.cc:205] Tried to move back new op lower bound from 7334099953275842560 to 7334099949315264512. Current Snapshot: MvccSnapshot[applied={T|T < 7334099953275842560}]
I20260927 23:29:06.736327 5700 raft_consensus.cc:482] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:06.774380 5700 raft_consensus.cc:504] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.788265 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "935c9a2e4d6c416db02443d2aaa57752" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:06.788893 5118 raft_consensus.cc:2454] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:06.790024 4910 leader_election.cc:302] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:06.790860 5681 raft_consensus.cc:2781] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:06.791159 5681 raft_consensus.cc:482] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:06.791429 5681 raft_consensus.cc:3037] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:06.791513 5718 raft_consensus.cc:482] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:06.791913 5718 raft_consensus.cc:504] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.793694 5718 leader_election.cc:288] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:06.794420 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "935c9a2e4d6c416db02443d2aaa57752" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
W20260927 23:29:06.796257 5051 leader_election.cc:341] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:06.798840 5681 raft_consensus.cc:504] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.800411 5700 leader_election.cc:288] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:06.800536 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "935c9a2e4d6c416db02443d2aaa57752" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:06.802196 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "935c9a2e4d6c416db02443d2aaa57752" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:06.804397 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "935c9a2e4d6c416db02443d2aaa57752" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:06.804963 5118 raft_consensus.cc:3037] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:06.813483 5118 raft_consensus.cc:2454] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:06.809881 5527 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:06.814618 5527 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.817945 5580 ts_tablet_manager.cc:1428] T be4adc02302b44c2b20a2be202ebe68d P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.293s user 0.005s sys 0.000s
I20260927 23:29:06.814970 4910 leader_election.cc:302] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a; no voters:
W20260927 23:29:06.822189 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:29:06.822420 5700 raft_consensus.cc:2781] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:06.822865 5700 raft_consensus.cc:686] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
I20260927 23:29:06.823616 5700 consensus_queue.cc:237] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:06.832705 4907 leader_election.cc:341] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Illegal state: must be running to vote when last-logged opid is not known
W20260927 23:29:06.833575 4907 leader_election.cc:341] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:06.834173 5681 leader_election.cc:288] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:06.836508 4971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "935c9a2e4d6c416db02443d2aaa57752" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:06.843055 5054 leader_election.cc:302] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:06.878700 5544 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 2 FOLLOWER]: Refusing update from remote peer f8460f73b40c41a28ec020fc04203564: Log matching property violated. Preceding OpId in replica: term: 1 index: 114. Preceding OpId from leader: term: 2 index: 146. (index mismatch)
I20260927 23:29:06.880132 5781 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 146, Last known committed idx: 114, Time since last communication: 0.000s
I20260927 23:29:06.886096 5788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:06.886534 5788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:06.889001 5788 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
I20260927 23:29:06.890373 4560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 146 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:29:06.893110 5543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" candidate_term: 3 candidate_status { last_received { term: 1 index: 146 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269" is_pre_election: true
I20260927 23:29:06.894456 5718 raft_consensus.cc:2726] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:06.894513 4624 catalog_manager.cc:3411] renamed table default.table_14122b0b934a4107ab5120525f201e5c [id=fa9d9f99dc5d4747a22f79e6a1533674] in HMS: new name default.table_143d2fa1ac844e81b1392bd3cc4d95c7
I20260927 23:29:06.895464 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:06.896588 5788 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260927 23:29:06.901247 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 58270.1us
I20260927 23:29:06.907371 5702 tablet_bootstrap.cc:492] T 935c9a2e4d6c416db02443d2aaa57752 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:06.907822 5702 ts_tablet_manager.cc:1397] T 935c9a2e4d6c416db02443d2aaa57752 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.417s user 0.008s sys 0.009s
I20260927 23:29:06.910440 5702 raft_consensus.cc:348] T 935c9a2e4d6c416db02443d2aaa57752 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.916431 5527 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:06.932832 4640 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 2 FOLLOWER]: Refusing update from remote peer f8460f73b40c41a28ec020fc04203564: Log matching property violated. Preceding OpId in replica: term: 1 index: 146. Preceding OpId from leader: term: 2 index: 146. (term mismatch)
I20260927 23:29:06.933185 4640 pending_rounds.cc:76] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Aborting all ops after (but not including) 145
I20260927 23:29:06.933400 4640 pending_rounds.cc:97] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d: Aborting uncommitted WRITE_OP operation due to leader change: 1.146
W20260927 23:29:06.935196 4623 catalog_manager.cc:6530] Aborted: an error occurred while writing to the sys-catalog: Op aborted by new leader
I20260927 23:29:06.938185 5781 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 146, Last known committed idx: 145, Time since last communication: 0.000s
I20260927 23:29:06.943409 5788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8460f73b40c41a28ec020fc04203564. Latest consensus state: current_term: 2 leader_uuid: "f8460f73b40c41a28ec020fc04203564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:06.944171 5788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:29:06.911128 5702 raft_consensus.cc:374] T 935c9a2e4d6c416db02443d2aaa57752 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:06.972559 5702 raft_consensus.cc:729] T 935c9a2e4d6c416db02443d2aaa57752 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:06.973308 5702 consensus_queue.cc:260] T 935c9a2e4d6c416db02443d2aaa57752 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.975692 5702 ts_tablet_manager.cc:1428] T 935c9a2e4d6c416db02443d2aaa57752 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.068s user 0.001s sys 0.005s
I20260927 23:29:06.976640 5702 tablet_bootstrap.cc:492] T 340a88a1702d48be9a8373fb3c6d540c P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:06.984272 5788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f8460f73b40c41a28ec020fc04203564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:06.984974 5788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:29:06.985702 5718 raft_consensus.cc:482] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:06.986115 5718 raft_consensus.cc:504] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:06.989607 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4adc02302b44c2b20a2be202ebe68d" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:06.990238 4831 raft_consensus.cc:2454] T be4adc02302b44c2b20a2be202ebe68d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
I20260927 23:29:06.991386 5051 leader_election.cc:302] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:06.992138 5534 raft_consensus.cc:2781] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:06.992479 5534 raft_consensus.cc:482] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:06.992775 5534 raft_consensus.cc:3037] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:06.998407 5718 leader_election.cc:288] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:07.002635 5534 raft_consensus.cc:504] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:07.016990 5534 leader_election.cc:288] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:07.019106 5054 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 46454.3us
I20260927 23:29:07.019377 5819 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8460f73b40c41a28ec020fc04203564. Latest consensus state: current_term: 2 leader_uuid: "f8460f73b40c41a28ec020fc04203564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:07.020103 5819 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:29:07.031867 4971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4adc02302b44c2b20a2be202ebe68d" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:07.032612 4971 raft_consensus.cc:2454] T be4adc02302b44c2b20a2be202ebe68d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 0.
W20260927 23:29:07.045758 4661 debug-util.cc:398] Leaking SignalData structure 0x7b08000b5020 after lost signal to thread 4593
W20260927 23:29:07.046464 4661 debug-util.cc:398] Leaking SignalData structure 0x7b08000a2140 after lost signal to thread 4664
I20260927 23:29:07.055573 4971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4adc02302b44c2b20a2be202ebe68d" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:07.056185 4971 raft_consensus.cc:3037] T be4adc02302b44c2b20a2be202ebe68d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
W20260927 23:29:07.060971 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:29:07.061362 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:07.062129 5527 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:07.065415 4831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be4adc02302b44c2b20a2be202ebe68d" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:07.065948 4831 raft_consensus.cc:3037] T be4adc02302b44c2b20a2be202ebe68d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:07.066293 5819 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "f8460f73b40c41a28ec020fc04203564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:07.067060 5819 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 23:29:07.085528 4831 raft_consensus.cc:2454] T be4adc02302b44c2b20a2be202ebe68d P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:07.086776 5051 leader_election.cc:302] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, f5ec98b2972d44838fa273eb7a10d01a; no voters:
I20260927 23:29:07.087474 5718 raft_consensus.cc:2781] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:07.088117 5718 raft_consensus.cc:686] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Becoming Leader. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Running, Role: LEADER
I20260927 23:29:07.089206 5804 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:07.089318 5702 tablet_bootstrap.cc:654] T 340a88a1702d48be9a8373fb3c6d540c P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:07.089588 5804 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:07.091316 5804 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:07.088863 5718 consensus_queue.cc:237] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:07.103284 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:29:07.116982 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:29:07.117379 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:07.117981 5437 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:07.121011 4971 raft_consensus.cc:2454] T be4adc02302b44c2b20a2be202ebe68d P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5ec98b2972d44838fa273eb7a10d01a in term 1.
I20260927 23:29:07.144111 5702 tablet_bootstrap.cc:492] T 340a88a1702d48be9a8373fb3c6d540c P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:07.144569 5702 ts_tablet_manager.cc:1397] T 340a88a1702d48be9a8373fb3c6d540c P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.168s user 0.007s sys 0.003s
I20260927 23:29:07.146992 5702 raft_consensus.cc:348] T 340a88a1702d48be9a8373fb3c6d540c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:07.147735 5702 raft_consensus.cc:374] T 340a88a1702d48be9a8373fb3c6d540c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:07.147997 5702 raft_consensus.cc:729] T 340a88a1702d48be9a8373fb3c6d540c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:07.151912 2276 master-stress-test.cc:578] Killing test thread 1/10
I20260927 23:29:07.157976 5781 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "f8460f73b40c41a28ec020fc04203564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:07.158849 5781 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: This master's current role is: LEADER
I20260927 23:29:07.159595 5820 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "f8460f73b40c41a28ec020fc04203564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:07.163270 5820 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564 [sys.catalog]: This master's current role is: LEADER
I20260927 23:29:07.163607 5718 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:07.163300 5702 consensus_queue.cc:260] T 340a88a1702d48be9a8373fb3c6d540c P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:07.164489 5718 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:07.166460 5718 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:07.168619 5809 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:29:07.177672 5702 ts_tablet_manager.cc:1428] T 340a88a1702d48be9a8373fb3c6d540c P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.033s user 0.004s sys 0.001s
W20260927 23:29:07.167763 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:29:07.192380 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:29:07.193288 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:07.194167 5437 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:07.220422 4630 catalog_manager.cc:2685] deleted HMS catalog entry for table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]
W20260927 23:29:07.277376 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9f07e8a37404636bda765146b20fc60 consensus state has no leader
I20260927 23:29:07.319561 5809 catalog_manager.cc:669] Loaded metadata for table default.table_8a2ddb6dcbbc4b8bba970c3bf5c37848 [id=078228a392424e6c96b3e5dbc1a7380d]
I20260927 23:29:07.344493 5437 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:07.344944 5437 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:07.346760 5437 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:07.348768 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:29:07.349335 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:29:07.349663 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:07.350847 5534 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:07.355487 5809 catalog_manager.cc:669] Loaded metadata for table default.table_1c3db2bbcc0e488ea4b6637492ff4221 [id=09f1820cca0c454e928a02f0677e7c9d]
I20260927 23:29:07.375470 5809 catalog_manager.cc:669] Loaded metadata for table default.table_829e227cbcac4bc585663258666cd43a [id=70012e8fafc049d2833983e702dd05ba]
I20260927 23:29:07.379909 4830 raft_consensus.cc:3037] T 340a88a1702d48be9a8373fb3c6d540c P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:07.388315 5437 consensus_queue.cc:1035] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:07.394474 5809 catalog_manager.cc:669] Loaded metadata for table default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc]
I20260927 23:29:07.434859 5809 catalog_manager.cc:669] Loaded metadata for table default.table_ecd08889b3cb4f09b30c4588cdceda4b [id=aea1156e91a84d018912c856e3f89d34]
I20260927 23:29:07.447580 4626 catalog_manager.cc:2148] created HMS catalog entry for table [id=7c2f12f99baf45748b6996c97097b4a4]
W20260927 23:29:07.449590 4626 catalog_manager.cc:2176] Illegal state: an error occurred while writing to the sys-catalog: replica 83cd22a89b7846999c0e4dfab80ebe9d is not leader of this config: current role FOLLOWER
I20260927 23:29:07.463109 5790 consensus_queue.cc:1035] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:07.469962 5437 consensus_queue.cc:1035] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:07.499507 5809 catalog_manager.cc:669] Loaded metadata for table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]
I20260927 23:29:07.502990 5790 consensus_queue.cc:1035] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:07.542798 5809 catalog_manager.cc:669] Loaded metadata for table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]
I20260927 23:29:07.545073 5809 catalog_manager.cc:669] Loaded metadata for table default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]
W20260927 23:29:07.549090 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:07.564347 5819 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f8460f73b40c41a28ec020fc04203564)
I20260927 23:29:07.564759 5819 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:07.566381 5819 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:29:07.567811 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 3 candidate_status { last_received { term: 2 index: 146 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:29:07.626353 5809 catalog_manager.cc:669] Loaded metadata for table default.table_14122b0b934a4107ab5120525f201e5c [id=fa9d9f99dc5d4747a22f79e6a1533674]
I20260927 23:29:07.678112 4831 raft_consensus.cc:3037] T 935c9a2e4d6c416db02443d2aaa57752 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:07.683431 5437 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:07.687672 5437 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:07.690208 5681 consensus_queue.cc:1035] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:29:07.694643 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:29:07.695218 5437 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:07.696729 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:29:07.697165 5051 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:07.698344 5437 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:07.742702 5700 consensus_queue.cc:1035] T 935c9a2e4d6c416db02443d2aaa57752 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:07.755025 5437 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:07.755514 5437 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:07.757388 5437 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:07.761025 4625 catalog_manager.cc:2148] created HMS catalog entry for table [id=51e8072f85474f71bc8086cc8918098a]
W20260927 23:29:07.762285 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:29:07.762974 4625 catalog_manager.cc:2176] Illegal state: an error occurred while writing to the sys-catalog: replica 83cd22a89b7846999c0e4dfab80ebe9d is not leader of this config: current role FOLLOWER
W20260927 23:29:07.764557 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:29:07.764961 5051 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:07.769977 5437 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:07.798408 4627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_e2a4264ee2df4a97a99f0846c2489d0b in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:29:07.802989 4629 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35136 (request call id 89) took 4998 ms. Trace:
I20260927 23:29:07.803326 4629 rpcz_store.cc:276] 0927 23:29:02.804773 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:02.805254 (+ 481us) service_pool.cc:227] Handling call
0927 23:29:04.484473 (+1679219us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:29:07.800435 (+3315962us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:29:07.802065 (+ 1630us) catalog_manager.cc:3616] Apply alter schema
0927 23:29:07.802924 (+ 859us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":78,"Hive Metastore.run_cpu_time_us":1747,"Hive Metastore.run_wall_time_us":1440592,"mutex_wait_us":27546}
I20260927 23:29:07.804466 4628 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35136 (request call id 90) took 4979 ms. Trace:
I20260927 23:29:07.804771 4628 rpcz_store.cc:276] 0927 23:29:02.825367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:02.827832 (+ 2465us) service_pool.cc:227] Handling call
0927 23:29:06.215573 (+3387741us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:29:07.804406 (+1588833us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":6829,"Hive Metastore.run_cpu_time_us":332,"Hive Metastore.run_wall_time_us":376609}
I20260927 23:29:07.807668 4624 rpcz_store.cc:275] Call kudu.master.MasterService.AlterTable from 127.0.0.1:35136 (request call id 91) took 4982 ms. Trace:
I20260927 23:29:07.808465 4630 rpcz_store.cc:275] Call kudu.master.MasterService.DeleteTable from 127.0.0.1:35136 (request call id 92) took 4982 ms. Trace:
I20260927 23:29:07.808768 4630 rpcz_store.cc:276] 0927 23:29:02.825908 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:02.826009 (+ 101us) service_pool.cc:227] Handling call
0927 23:29:06.216962 (+3390953us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:29:07.808410 (+1591448us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":676643,"Hive Metastore.run_cpu_time_us":310,"Hive Metastore.run_wall_time_us":322417}
I20260927 23:29:07.807960 4624 rpcz_store.cc:276] 0927 23:29:02.825649 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:02.826037 (+ 388us) service_pool.cc:227] Handling call
0927 23:29:06.216013 (+3389976us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:29:07.805855 (+1589842us) catalog_manager.cc:2347] Looking up, locking, and authorizing table
0927 23:29:07.806761 (+ 906us) catalog_manager.cc:3616] Apply alter schema
0927 23:29:07.807607 (+ 846us) inbound_call.cc:173] Queueing success response
Metrics: {"Hive Metastore.queue_time_us":382907,"Hive Metastore.run_cpu_time_us":1679,"Hive Metastore.run_wall_time_us":294360,"spinlock_wait_cycles":163328}
I20260927 23:29:07.871789 5809 tablet_loader.cc:96] loaded metadata for tablet 0e0aaa4d2fc8491aaf77eda6aa94616f (table default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc])
I20260927 23:29:07.873347 5809 tablet_loader.cc:96] loaded metadata for tablet 1572bbcfebce4d218b1a0f38ecab0384 (table default.table_ecd08889b3cb4f09b30c4588cdceda4b [id=aea1156e91a84d018912c856e3f89d34])
I20260927 23:29:07.874879 5809 tablet_loader.cc:96] loaded metadata for tablet 1c89dab9709541c1afc6a8cdcd43a118 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d])
I20260927 23:29:07.876121 5809 tablet_loader.cc:96] loaded metadata for tablet 340a88a1702d48be9a8373fb3c6d540c (table default.table_829e227cbcac4bc585663258666cd43a [id=70012e8fafc049d2833983e702dd05ba])
I20260927 23:29:07.877341 5809 tablet_loader.cc:96] loaded metadata for tablet 8546d6e754784ff886f421eb0ddb796a (table default.table_1c3db2bbcc0e488ea4b6637492ff4221 [id=09f1820cca0c454e928a02f0677e7c9d])
I20260927 23:29:07.941282 5809 tablet_loader.cc:96] loaded metadata for tablet 935c9a2e4d6c416db02443d2aaa57752 (table default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40])
I20260927 23:29:07.942564 5809 tablet_loader.cc:96] loaded metadata for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863])
I20260927 23:29:07.943764 5809 tablet_loader.cc:96] loaded metadata for tablet be4adc02302b44c2b20a2be202ebe68d (table default.table_8a2ddb6dcbbc4b8bba970c3bf5c37848 [id=078228a392424e6c96b3e5dbc1a7380d])
I20260927 23:29:07.949381 5437 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:07.949824 5437 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:07.951646 5437 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:07.954478 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:29:07.961860 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:29:07.962249 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:07.964395 5437 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:07.971514 5437 consensus_queue.cc:1035] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:07.989176 5809 tablet_loader.cc:96] loaded metadata for tablet fe9cb9be216d42fa81948f5d9da3d4d8 (table default.table_14122b0b934a4107ab5120525f201e5c [id=fa9d9f99dc5d4747a22f79e6a1533674])
I20260927 23:29:07.993027 5437 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:07.993570 5437 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:07.996923 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:29:07.997568 5437 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:08.007382 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:29:08.007782 5051 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:08.012169 5437 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:08.058864 4560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 3 candidate_status { last_received { term: 2 index: 146 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:29:08.060715 5471 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d9ed71d97661411891f2d2de4cdc2269; no voters: 83cd22a89b7846999c0e4dfab80ebe9d, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:08.061707 5819 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260927 23:29:08.058825 5809 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:29:08.097625 4621 catalog_manager.cc:2148] created HMS catalog entry for table [id=572006748eda47c2bd6cdfd3142d5575]
W20260927 23:29:08.099596 4621 catalog_manager.cc:2176] Illegal state: an error occurred while writing to the sys-catalog: replica 83cd22a89b7846999c0e4dfab80ebe9d is not leader of this config: current role FOLLOWER
I20260927 23:29:08.144598 5534 consensus_queue.cc:1035] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:08.159631 5804 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:08.160092 5804 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:08.163658 5804 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:08.172811 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:29:08.198642 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:08.199054 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:08.199712 5804 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:08.243254 5809 catalog_manager.cc:1259] Loaded cluster ID: 6dc11a7b873d4b11830ae46041eb6de8
I20260927 23:29:08.245575 5809 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 23:29:08.311142 5809 catalog_manager.cc:1504] Loading token signing keys...
I20260927 23:29:08.332348 5527 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:08.332845 5527 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:08.334852 5527 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:08.354521 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:29:08.363685 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:08.364100 5051 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:08.366482 5527 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:29:08.407389 5437 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:08.407814 5437 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:08.409591 5437 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:08.412222 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:08.429903 5809 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: Loaded TSK: 0
I20260927 23:29:08.432134 5809 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 23:29:08.448393 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:08.448804 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:08.449899 5527 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:08.598187 4626 catalog_manager.cc:2158] deleted orphaned HMS catalog entry for table [id=7c2f12f99baf45748b6996c97097b4a4]
I20260927 23:29:08.599086 4626 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35136 (request call id 105) took 4471 ms. Trace:
I20260927 23:29:08.599416 4626 rpcz_store.cc:276] 0927 23:29:04.127934 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:04.146815 (+ 18881us) service_pool.cc:227] Handling call
0927 23:29:06.217702 (+2070887us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:29:06.221224 (+ 3522us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:07.447534 (+1226310us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:07.449355 (+ 1821us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:07.449379 (+ 24us) write_op.cc:454] Released schema lock
0927 23:29:07.449875 (+ 496us) catalog_manager.cc:2155] Rolling back HMS table creation
0927 23:29:08.599025 (+1149150us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:07.449132 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 83cd22a89b7846999c0e4dfab80ebe9d is not leader of this config: current role FOLLOWER)
0927 23:29:07.449174 (+ 42us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:07.449198 (+ 24us) write_op.cc:454] Released schema lock
0927 23:29:07.449220 (+ 22us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1642441,"Hive Metastore.run_cpu_time_us":934,"Hive Metastore.run_wall_time_us":730316,"child_traces":[["op",{}]]}
W20260927 23:29:08.602566 5177 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.57.60:33749)
W20260927 23:29:08.657213 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:08.618857 5809 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
W20260927 23:29:08.713303 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 6: this message will repeat every 5th retry.
W20260927 23:29:08.713871 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:08.747202 5437 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:08.747622 5437 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:08.750488 5437 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:08.753137 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:29:08.765342 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 327adcc2a27f42acb3a77d49fbf73796 consensus state has no leader
W20260927 23:29:08.768694 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:29:08.769210 5051 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:08.770777 5804 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:08.817186 5847 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f8460f73b40c41a28ec020fc04203564)
I20260927 23:29:08.817763 5847 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:08.819957 5847 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
I20260927 23:29:08.820851 5544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 146 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269" is_pre_election: true
I20260927 23:29:08.823338 4560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 146 } } ignore_live_leader: false dest_uuid: "f8460f73b40c41a28ec020fc04203564" is_pre_election: true
I20260927 23:29:08.826257 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:08.834797 5847 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260927 23:29:08.888932 5809 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: 16
W20260927 23:29:08.904719 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:08.934580 4551 master_service.cc:946] ReplaceTablet: received request to replace tablet 48dcf44bb7b744b9a98626dbfb6f4e6c from {username='slave'} at 127.0.0.1:42536
I20260927 23:29:08.940231 4551 master_service.cc:946] ReplaceTablet: received request to replace tablet 0e0aaa4d2fc8491aaf77eda6aa94616f from {username='slave'} at 127.0.0.1:42536
W20260927 23:29:08.995992 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
W20260927 23:29:08.996526 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:29:08.998792 5164 heartbeater.cc:503] Master 127.2.57.61:46527 was elected leader, sending a full tablet report...
W20260927 23:29:09.040556 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
W20260927 23:29:09.040932 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:29:09.041460 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet b373bde60f734d049c4a45255e2b2e2a
W20260927 23:29:09.041774 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
W20260927 23:29:09.041989 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet c739222645654eb39285a2ad1b6c35f5
W20260927 23:29:09.042191 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:29:09.042404 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 309465f419c24549bf5416a4503b9299
W20260927 23:29:09.042850 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet df6ea36da9f6455bb399c2229dcb3790
W20260927 23:29:09.043109 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:29:09.043339 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:29:09.043623 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
W20260927 23:29:09.043892 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
W20260927 23:29:09.044164 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
W20260927 23:29:09.044431 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
W20260927 23:29:09.044663 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
W20260927 23:29:09.046660 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
W20260927 23:29:09.047029 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:29:09.047271 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
W20260927 23:29:09.047506 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
W20260927 23:29:09.047822 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet b4ec12a496904c3599bdc001476ebee9
W20260927 23:29:09.048210 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet f171cc1f0094480a9cc6357eab93dff1
W20260927 23:29:09.048466 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
I20260927 23:29:08.946048 4552 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42536:
name: "default.Table_85c5c1d72c47496b94eb0f8c080ead12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:29:09.052059 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
W20260927 23:29:09.052385 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:29:09.052634 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
W20260927 23:29:09.052879 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
I20260927 23:29:09.094281 4738 catalog_manager.cc:2709] Deleting table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d] in response to Hive Metastore notification log event 17
W20260927 23:29:09.124044 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:09.124323 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
W20260927 23:29:09.124516 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
W20260927 23:29:09.124688 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
W20260927 23:29:09.140578 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet e56378a5d42e47d395c881a0898da495
W20260927 23:29:09.140969 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
W20260927 23:29:09.141302 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet e4e5cb0d645846159e09ab71a897f6df
W20260927 23:29:09.141733 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:29:09.143524 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:29:09.143812 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
W20260927 23:29:09.144063 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:29:09.144330 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
W20260927 23:29:09.145239 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet c5552715138046279f00dcf497d05a72
W20260927 23:29:09.145512 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:29:09.145748 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
W20260927 23:29:09.145978 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
W20260927 23:29:09.146209 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 9912e4dd70ba482398fbd831a8242f12
W20260927 23:29:09.146468 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:29:09.146690 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:29:09.147105 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:29:09.147322 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:29:09.147552 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:29:09.147964 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
W20260927 23:29:09.148216 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:29:09.172919 4625 catalog_manager.cc:2158] deleted orphaned HMS catalog entry for table [id=51e8072f85474f71bc8086cc8918098a]
I20260927 23:29:09.173741 4625 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35136 (request call id 103) took 5335 ms. Trace:
I20260927 23:29:09.174063 4625 rpcz_store.cc:276] 0927 23:29:03.838681 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:03.886423 (+ 47742us) service_pool.cc:227] Handling call
0927 23:29:06.221709 (+2335286us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:29:06.223382 (+ 1673us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:07.760983 (+1537601us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:07.762745 (+ 1762us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:07.762770 (+ 25us) write_op.cc:454] Released schema lock
0927 23:29:07.763287 (+ 517us) catalog_manager.cc:2155] Rolling back HMS table creation
0927 23:29:09.173681 (+1410394us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:07.762544 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 83cd22a89b7846999c0e4dfab80ebe9d is not leader of this config: current role FOLLOWER)
0927 23:29:07.762580 (+ 36us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:07.762602 (+ 22us) write_op.cc:454] Released schema lock
0927 23:29:07.762624 (+ 22us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":2057529,"Hive Metastore.run_cpu_time_us":879,"Hive Metastore.run_wall_time_us":888018,"child_traces":[["op",{}]]}
I20260927 23:29:09.189715 4551 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: Sending DeleteTablet for 3 replicas of tablet 0e0aaa4d2fc8491aaf77eda6aa94616f
I20260927 23:29:09.193449 4551 catalog_manager.cc:6565] ReplaceTablet: tablet 0e0aaa4d2fc8491aaf77eda6aa94616f deleted and replaced by tablet bd53bce1e3de426fbe05fd3225aae88d
I20260927 23:29:09.193642 4550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42536:
name: "default.Table_1fbfdf99ff4e4165bfcf02ceb0247c87"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:29:09.241431 4627 catalog_manager.cc:2148] created HMS catalog entry for table [id=841dea283b0d475ebee6bc0637c75421]
I20260927 23:29:09.242231 5437 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:09.242635 5437 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
W20260927 23:29:09.243175 4627 catalog_manager.cc:2176] Illegal state: an error occurred while writing to the sys-catalog: replica 83cd22a89b7846999c0e4dfab80ebe9d is not leader of this config: current role FOLLOWER
W20260927 23:29:09.247902 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:29:09.260878 5437 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:09.274562 4549 catalog_manager.cc:5495] T 340a88a1702d48be9a8373fb3c6d540c P f5ec98b2972d44838fa273eb7a10d01a reported cstate change: term changed from 0 to 1, leader changed from <none> to f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3). New cstate: current_term: 1 leader_uuid: "f5ec98b2972d44838fa273eb7a10d01a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } health_report { overall_health: HEALTHY } } }
I20260927 23:29:09.292044 5020 heartbeater.cc:503] Master 127.2.57.61:46527 was elected leader, sending a full tablet report...
I20260927 23:29:09.298159 4549 catalog_manager.cc:5495] T 935c9a2e4d6c416db02443d2aaa57752 P f5ec98b2972d44838fa273eb7a10d01a reported cstate change: term changed from 0 to 1, leader changed from <none> to bad92a10de2c4459827cdbb18854d76d (127.2.57.2). New cstate: current_term: 1 leader_uuid: "bad92a10de2c4459827cdbb18854d76d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } }
W20260927 23:29:09.317386 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:29:09.316881 4549 catalog_manager.cc:5495] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e62b4926a8448b2b5f4923553328bca (127.2.57.1). New cstate: current_term: 1 leader_uuid: "7e62b4926a8448b2b5f4923553328bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } }
I20260927 23:29:09.317824 5051 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:09.318243 4549 catalog_manager.cc:5587] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519) does not have the latest schema for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]). Expected version 1 got 0
I20260927 23:29:09.320036 5527 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:09.327324 4549 catalog_manager.cc:5495] T be4adc02302b44c2b20a2be202ebe68d P f5ec98b2972d44838fa273eb7a10d01a reported cstate change: term changed from 0 to 1, leader changed from <none> to f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3). New cstate: current_term: 1 leader_uuid: "f5ec98b2972d44838fa273eb7a10d01a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } health_report { overall_health: HEALTHY } } }
I20260927 23:29:09.333860 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0aaa4d2fc8491aaf77eda6aa94616f with delete_type TABLET_DATA_DELETED (replaced by tablet bd53bce1e3de426fbe05fd3225aae88d) from {username='slave'} at 127.0.0.1:35112
W20260927 23:29:09.334232 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:09.337106 4738 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: Sending DeleteTablet for 3 replicas of tablet 1c89dab9709541c1afc6a8cdcd43a118
I20260927 23:29:09.343881 4811 tablet_service.cc:1467] Processing CreateTablet for tablet bd53bce1e3de426fbe05fd3225aae88d (DEFAULT_TABLE table=default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:09.345005 4811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd53bce1e3de426fbe05fd3225aae88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:09.353071 5865 tablet_replica.cc:330] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:09.356833 5095 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0aaa4d2fc8491aaf77eda6aa94616f with delete_type TABLET_DATA_DELETED (replaced by tablet bd53bce1e3de426fbe05fd3225aae88d) from {username='slave'} at 127.0.0.1:48848
I20260927 23:29:09.359805 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e0aaa4d2fc8491aaf77eda6aa94616f with delete_type TABLET_DATA_DELETED (replaced by tablet bd53bce1e3de426fbe05fd3225aae88d) from {username='slave'} at 127.0.0.1:41168
I20260927 23:29:09.361406 5098 tablet_service.cc:1467] Processing CreateTablet for tablet bd53bce1e3de426fbe05fd3225aae88d (DEFAULT_TABLE table=default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:09.361869 4951 tablet_service.cc:1467] Processing CreateTablet for tablet bd53bce1e3de426fbe05fd3225aae88d (DEFAULT_TABLE table=default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:09.362414 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd53bce1e3de426fbe05fd3225aae88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:09.363057 4951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd53bce1e3de426fbe05fd3225aae88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:09.367172 4953 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c89dab9709541c1afc6a8cdcd43a118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:29:09 UTC) from {username='slave'} at 127.0.0.1:41168
I20260927 23:29:09.372506 5097 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c89dab9709541c1afc6a8cdcd43a118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:29:09 UTC) from {username='slave'} at 127.0.0.1:48848
I20260927 23:29:09.397769 5866 tablet_replica.cc:330] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a: stopping tablet replica
I20260927 23:29:09.398869 5866 raft_consensus.cc:2229] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:09.399712 5866 raft_consensus.cc:2258] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:09.411427 5867 tablet_replica.cc:330] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
W20260927 23:29:09.418196 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
W20260927 23:29:09.418462 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet df6ea36da9f6455bb399c2229dcb3790
W20260927 23:29:09.418634 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
W20260927 23:29:09.418799 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:09.422287 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
W20260927 23:29:09.422559 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
W20260927 23:29:09.423151 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
W20260927 23:29:09.423470 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
W20260927 23:29:09.423684 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
W20260927 23:29:09.423887 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:29:09.424078 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
W20260927 23:29:09.424286 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
W20260927 23:29:09.424500 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
W20260927 23:29:09.424696 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:29:09.424896 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
W20260927 23:29:09.425089 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
W20260927 23:29:09.425340 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
W20260927 23:29:09.425544 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
W20260927 23:29:09.426090 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
W20260927 23:29:09.427508 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
W20260927 23:29:09.427879 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
W20260927 23:29:09.428380 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
W20260927 23:29:09.428602 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:29:09.431600 5867 raft_consensus.cc:2229] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Raft consensus shutting down.
W20260927 23:29:09.434146 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:29:09.434517 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
W20260927 23:29:09.434794 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
W20260927 23:29:09.435050 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
W20260927 23:29:09.435539 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
W20260927 23:29:09.435968 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:29:09.436203 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
I20260927 23:29:09.432608 5866 ts_tablet_manager.cc:1905] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:09.437414 5867 raft_consensus.cc:2258] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:09.443164 4551 catalog_manager.cc:5587] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717) does not have the latest schema for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]). Expected version 1 got 0
I20260927 23:29:09.354552 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet 1c89dab9709541c1afc6a8cdcd43a118 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:29:09 UTC) from {username='slave'} at 127.0.0.1:35112
I20260927 23:29:09.457237 5865 raft_consensus.cc:2229] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:09.459528 5865 raft_consensus.cc:2258] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:09.462337 5867 ts_tablet_manager.cc:1905] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:09.473120 4738 catalog_manager.cc:2709] Deleting table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863] in response to Hive Metastore notification log event 19
I20260927 23:29:09.479308 4549 catalog_manager.cc:6040] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: default.table_ecd08889b3cb4f09b30c4588cdceda4b [id=aea1156e91a84d018912c856e3f89d34] alter complete (version 1)
I20260927 23:29:09.486927 5865 ts_tablet_manager.cc:1905] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:29:09.508592 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
W20260927 23:29:09.508872 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
W20260927 23:29:09.509054 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
W20260927 23:29:09.509218 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
W20260927 23:29:09.509384 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
W20260927 23:29:09.509546 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
W20260927 23:29:09.509927 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
W20260927 23:29:09.510210 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:29:09.510389 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
W20260927 23:29:09.510560 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
W20260927 23:29:09.510743 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
W20260927 23:29:09.510913 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
W20260927 23:29:09.524904 4910 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:45519, user_credentials={real_user=slave}} blocked reactor thread for 51091.7us
I20260927 23:29:09.528435 5868 tablet_bootstrap.cc:492] T bd53bce1e3de426fbe05fd3225aae88d P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:09.533579 5869 tablet_bootstrap.cc:492] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
W20260927 23:29:09.559608 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
I20260927 23:29:09.575160 4738 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: Sending DeleteTablet for 3 replicas of tablet b9f07e8a37404636bda765146b20fc60
I20260927 23:29:09.575846 5868 tablet_bootstrap.cc:654] T bd53bce1e3de426fbe05fd3225aae88d P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:09.577598 4810 tablet_service.cc:1514] Processing DeleteTablet for tablet b9f07e8a37404636bda765146b20fc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:29:09 UTC) from {username='slave'} at 127.0.0.1:35112
I20260927 23:29:09.579355 4879 heartbeater.cc:503] Master 127.2.57.61:46527 was elected leader, sending a full tablet report...
I20260927 23:29:09.582007 5098 tablet_service.cc:1514] Processing DeleteTablet for tablet b9f07e8a37404636bda765146b20fc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:29:09 UTC) from {username='slave'} at 127.0.0.1:48848
I20260927 23:29:09.582301 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet b9f07e8a37404636bda765146b20fc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 23:29:09 UTC) from {username='slave'} at 127.0.0.1:41168
I20260927 23:29:09.586468 4738 catalog_manager.cc:2709] Deleting table default.table_85c5c1d72c47496b94eb0f8c080ead12 [id=7c2f12f99baf45748b6996c97097b4a4] in response to Hive Metastore notification log event 23
W20260927 23:29:09.587339 4738 hms_notification_log_listener.cc:294] Failed to handle Hive Metastore notification: 23 DROP_TABLE default.table_85c5c1d72c47496b94eb0f8c080ead12: Not found: the table does not exist: table_id: "7c2f12f99baf45748b6996c97097b4a4" table_name: "default.table_85c5c1d72c47496b94eb0f8c080ead12"
W20260927 23:29:09.590525 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 0e0aaa4d2fc8491aaf77eda6aa94616f
W20260927 23:29:09.590744 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
I20260927 23:29:09.590884 5869 tablet_bootstrap.cc:654] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:09.591318 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:09.591599 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
W20260927 23:29:09.591876 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
W20260927 23:29:09.592036 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
W20260927 23:29:09.592198 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
W20260927 23:29:09.607456 4552 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_85c5c1d72c47496b94eb0f8c080ead12 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 23:29:09.633320 5877 tablet_bootstrap.cc:492] T bd53bce1e3de426fbe05fd3225aae88d P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:09.646117 4551 catalog_manager.cc:5587] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571) does not have the latest schema for tablet fe9cb9be216d42fa81948f5d9da3d4d8 (table default.table_143d2fa1ac844e81b1392bd3cc4d95c7 [id=fa9d9f99dc5d4747a22f79e6a1533674]). Expected version 1 got 0
W20260927 23:29:09.652263 4549 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
I20260927 23:29:09.660624 4738 catalog_manager.cc:2709] Deleting table default.table_1fbfdf99ff4e4165bfcf02ceb0247c87 [id=51e8072f85474f71bc8086cc8918098a] in response to Hive Metastore notification log event 24
W20260927 23:29:09.661616 4738 hms_notification_log_listener.cc:294] Failed to handle Hive Metastore notification: 24 DROP_TABLE default.table_1fbfdf99ff4e4165bfcf02ceb0247c87: Not found: the table does not exist: table_id: "51e8072f85474f71bc8086cc8918098a" table_name: "default.table_1fbfdf99ff4e4165bfcf02ceb0247c87"
I20260927 23:29:09.685916 5804 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:09.686384 5804 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:09.693264 4550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table default.table_1fbfdf99ff4e4165bfcf02ceb0247c87 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260927 23:29:09.696748 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:29:09.697628 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:29:09.697945 5051 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:09.702951 5527 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:09.688133 5804 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:09.712271 5893 tablet.cc:1722] T fe9cb9be216d42fa81948f5d9da3d4d8 P f5ec98b2972d44838fa273eb7a10d01a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:29:09.735380 5868 tablet_bootstrap.cc:492] T bd53bce1e3de426fbe05fd3225aae88d P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:09.738927 5868 ts_tablet_manager.cc:1397] T bd53bce1e3de426fbe05fd3225aae88d P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.211s user 0.011s sys 0.000s
I20260927 23:29:09.741503 5868 raft_consensus.cc:348] T bd53bce1e3de426fbe05fd3225aae88d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:09.730655 5891 tablet.cc:1722] T fe9cb9be216d42fa81948f5d9da3d4d8 P 7e62b4926a8448b2b5f4923553328bca: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:29:09.738865 5869 tablet_bootstrap.cc:492] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:09.744352 5877 tablet_bootstrap.cc:654] T bd53bce1e3de426fbe05fd3225aae88d P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:09.744810 5869 ts_tablet_manager.cc:1397] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.212s user 0.011s sys 0.000s
W20260927 23:29:09.747196 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
W20260927 23:29:09.747466 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
W20260927 23:29:09.747641 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:29:09.753365 4621 catalog_manager.cc:2158] deleted orphaned HMS catalog entry for table [id=572006748eda47c2bd6cdfd3142d5575]
I20260927 23:29:09.754248 4621 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35136 (request call id 97) took 6557 ms. Trace:
I20260927 23:29:09.754537 4621 rpcz_store.cc:276] 0927 23:29:03.196552 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:03.201402 (+ 4850us) service_pool.cc:227] Handling call
0927 23:29:06.218999 (+3017597us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:29:06.225721 (+ 6722us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:08.097587 (+1871866us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:08.099367 (+ 1780us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:08.099390 (+ 23us) write_op.cc:454] Released schema lock
0927 23:29:08.099904 (+ 514us) catalog_manager.cc:2155] Rolling back HMS table creation
0927 23:29:09.754122 (+1654218us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:08.099129 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 83cd22a89b7846999c0e4dfab80ebe9d is not leader of this config: current role FOLLOWER)
0927 23:29:08.099164 (+ 35us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:08.099186 (+ 22us) write_op.cc:454] Released schema lock
0927 23:29:08.099227 (+ 41us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":2710885,"Hive Metastore.run_cpu_time_us":919,"Hive Metastore.run_wall_time_us":813498,"child_traces":[["op",{}]]}
I20260927 23:29:09.756824 5869 raft_consensus.cc:348] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:09.758103 5177 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.57.61:46527)
I20260927 23:29:09.762821 5869 raft_consensus.cc:374] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:09.763688 5869 raft_consensus.cc:729] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:09.765326 5869 consensus_queue.cc:260] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:09.764465 5868 raft_consensus.cc:374] T bd53bce1e3de426fbe05fd3225aae88d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:09.766490 5868 raft_consensus.cc:729] T bd53bce1e3de426fbe05fd3225aae88d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
I20260927 23:29:09.767424 4549 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42536:
name: "default.Table_68574d2015dd47bbb92137d783bad572"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:29:09.746986 5866 ts_tablet_manager.cc:1918] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:29:09.773059 5866 log.cc:1196] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/0e0aaa4d2fc8491aaf77eda6aa94616f
W20260927 23:29:09.773460 4548 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:29:09.775936 4769 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.2.57.3:45519, user_credentials={real_user=slave}} blocked reactor thread for 62208.1us
I20260927 23:29:09.776752 5896 tablet.cc:1722] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:29:09.778622 5866 ts_tablet_manager.cc:1939] T 0e0aaa4d2fc8491aaf77eda6aa94616f P f5ec98b2972d44838fa273eb7a10d01a: Deleting consensus metadata
I20260927 23:29:09.780498 5867 ts_tablet_manager.cc:1918] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:29:09.783466 4538 catalog_manager.cc:4841] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): tablet 0e0aaa4d2fc8491aaf77eda6aa94616f (table default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc]) successfully deleted
I20260927 23:29:09.784457 5867 log.cc:1196] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/0e0aaa4d2fc8491aaf77eda6aa94616f
I20260927 23:29:09.787732 5865 ts_tablet_manager.cc:1918] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260927 23:29:09.789295 5865 log.cc:1196] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/0e0aaa4d2fc8491aaf77eda6aa94616f
I20260927 23:29:09.790313 5867 ts_tablet_manager.cc:1939] T 0e0aaa4d2fc8491aaf77eda6aa94616f P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:09.792781 5866 tablet_replica.cc:330] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a: stopping tablet replica
I20260927 23:29:09.793787 5866 raft_consensus.cc:2229] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:29:09.806679 5898 tablet.cc:1722] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260927 23:29:09.807682 4548 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
I20260927 23:29:09.809679 5899 tablet.cc:1722] T fe9cb9be216d42fa81948f5d9da3d4d8 P bad92a10de2c4459827cdbb18854d76d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260927 23:29:09.811755 5866 raft_consensus.cc:2258] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:09.814291 4538 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet 0e0aaa4d2fc8491aaf77eda6aa94616f (table default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc]) successfully deleted
I20260927 23:29:09.815198 5867 tablet_replica.cc:330] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:09.816912 5866 ts_tablet_manager.cc:1905] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:09.819954 5877 tablet_bootstrap.cc:492] T bd53bce1e3de426fbe05fd3225aae88d P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:09.820621 5877 ts_tablet_manager.cc:1397] T bd53bce1e3de426fbe05fd3225aae88d P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.188s user 0.009s sys 0.002s
I20260927 23:29:09.823328 5877 raft_consensus.cc:348] T bd53bce1e3de426fbe05fd3225aae88d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:09.827561 5877 raft_consensus.cc:374] T bd53bce1e3de426fbe05fd3225aae88d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:09.828052 5877 raft_consensus.cc:729] T bd53bce1e3de426fbe05fd3225aae88d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:09.817827 5865 ts_tablet_manager.cc:1939] T 0e0aaa4d2fc8491aaf77eda6aa94616f P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:09.838907 5869 ts_tablet_manager.cc:1428] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.094s user 0.006s sys 0.000s
I20260927 23:29:09.840898 5867 raft_consensus.cc:2229] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:09.841572 5867 raft_consensus.cc:2258] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:09.843544 5867 ts_tablet_manager.cc:1905] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:29:09.856736 4548 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:29:09.860579 5881 raft_consensus.cc:482] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:09.861038 5881 raft_consensus.cc:504] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:09.863401 5881 leader_election.cc:288] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:09.866122 5865 tablet_replica.cc:330] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:09.866334 4535 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 0e0aaa4d2fc8491aaf77eda6aa94616f (table default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc]) successfully deleted
I20260927 23:29:09.866958 5865 raft_consensus.cc:2229] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:09.867576 5865 raft_consensus.cc:2258] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:09.818370 5897 tablet.cc:1722] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
W20260927 23:29:09.884670 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:09.892239 4551 catalog_manager.cc:6040] T 00000000000000000000000000000000 P f8460f73b40c41a28ec020fc04203564: default.table_143d2fa1ac844e81b1392bd3cc4d95c7 [id=fa9d9f99dc5d4747a22f79e6a1533674] alter complete (version 1)
I20260927 23:29:09.828855 5877 consensus_queue.cc:260] T bd53bce1e3de426fbe05fd3225aae88d P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:09.911311 4966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd53bce1e3de426fbe05fd3225aae88d" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:09.911944 4966 raft_consensus.cc:2454] T bd53bce1e3de426fbe05fd3225aae88d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 0.
I20260927 23:29:09.777071 5868 consensus_queue.cc:260] T bd53bce1e3de426fbe05fd3225aae88d P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:09.914644 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd53bce1e3de426fbe05fd3225aae88d" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:09.919483 5115 raft_consensus.cc:2454] T bd53bce1e3de426fbe05fd3225aae88d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 0.
I20260927 23:29:09.923857 4769 leader_election.cc:302] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:09.925262 5881 raft_consensus.cc:2781] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:09.917491 5868 ts_tablet_manager.cc:1428] T bd53bce1e3de426fbe05fd3225aae88d P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.178s user 0.006s sys 0.000s
I20260927 23:29:09.925580 5881 raft_consensus.cc:482] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:09.925853 5881 raft_consensus.cc:3037] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:09.932282 5881 raft_consensus.cc:504] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:09.935053 5881 leader_election.cc:288] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Requested vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:09.936424 5877 ts_tablet_manager.cc:1428] T bd53bce1e3de426fbe05fd3225aae88d P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.115s user 0.006s sys 0.000s
I20260927 23:29:09.937148 4966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd53bce1e3de426fbe05fd3225aae88d" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:09.937665 4966 raft_consensus.cc:3037] T bd53bce1e3de426fbe05fd3225aae88d P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:09.946048 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd53bce1e3de426fbe05fd3225aae88d" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:09.947407 5115 raft_consensus.cc:3037] T bd53bce1e3de426fbe05fd3225aae88d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:09.952137 5865 ts_tablet_manager.cc:1905] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:29:09.958529 4548 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
I20260927 23:29:09.959916 4966 raft_consensus.cc:2454] T bd53bce1e3de426fbe05fd3225aae88d P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 1.
I20260927 23:29:10.004454 4769 leader_election.cc:302] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:09.988811 4552 catalog_manager.cc:2148] created HMS catalog entry for table [id=496c618bcbd04643b08992de376c904b]
I20260927 23:29:10.009742 5881 raft_consensus.cc:2781] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:10.010244 5881 raft_consensus.cc:686] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: Becoming Leader. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Running, Role: LEADER
I20260927 23:29:10.011058 5881 consensus_queue.cc:237] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:10.048652 4548 catalog_manager.cc:5328] Ignoring report from unknown tablet df6ea36da9f6455bb399c2229dcb3790
I20260927 23:29:10.064445 4552 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:42536 (request call id 34) took 1119 ms. Trace:
I20260927 23:29:10.065088 4552 rpcz_store.cc:276] 0927 23:29:08.945183 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:08.945302 (+ 119us) service_pool.cc:227] Handling call
0927 23:29:09.608046 (+662744us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:29:09.635686 (+ 27640us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:09.988761 (+353075us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:10.031111 (+ 42350us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:29:10.063947 (+ 32836us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:29:10.064361 (+ 414us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:10.007289 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:10.007695 (+ 406us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:10.007721 (+ 26us) write_op.cc:435] Acquired schema lock
0927 23:29:10.007736 (+ 15us) tablet.cc:660] Decoding operations
0927 23:29:10.008057 (+ 321us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:10.008098 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:10.008123 (+ 25us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:29:10.008363 (+ 240us) tablet.cc:699] Row locks acquired
0927 23:29:10.008379 (+ 16us) write_op.cc:260] PREPARE: finished
0927 23:29:10.008503 (+ 124us) write_op.cc:270] Start()
0927 23:29:10.008594 (+ 91us) write_op.cc:276] Timestamp: P: 1790551750008417 usec, L: 0
0927 23:29:10.008612 (+ 18us) op_driver.cc:331] REPLICATION: starting
0927 23:29:10.009182 (+ 570us) log.cc:842] Serialized 433 byte log entry
0927 23:29:10.021588 (+ 12406us) op_driver.cc:444] REPLICATION: finished
0927 23:29:10.021898 (+ 310us) write_op.cc:301] APPLY: starting
0927 23:29:10.021954 (+ 56us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:10.022064 (+ 110us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:10.022164 (+ 100us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:10.022847 (+ 683us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:10.022987 (+ 140us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:10.023031 (+ 44us) write_op.cc:312] APPLY: finished
0927 23:29:10.023216 (+ 185us) log.cc:842] Serialized 29 byte log entry
0927 23:29:10.023362 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:10.023434 (+ 72us) write_op.cc:454] Released schema lock
0927 23:29:10.023465 (+ 31us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:10.023628 (+ 163us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:10.023651 (+ 23us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":85,"Hive Metastore.run_cpu_time_us":803,"Hive Metastore.run_wall_time_us":351641,"child_traces":[["op",{"apply.queue_time_us":160,"apply.run_cpu_time_us":1701,"apply.run_wall_time_us":1900,"num_ops":2,"prepare.queue_time_us":109,"prepare.run_cpu_time_us":2225,"prepare.run_wall_time_us":2225,"raft.queue_time_us":3776,"raft.run_cpu_time_us":1455,"raft.run_wall_time_us":1455,"replication_time_us":12898}]]}
I20260927 23:29:10.078809 5866 ts_tablet_manager.cc:1918] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:29:10.080040 5866 log.cc:1196] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/1c89dab9709541c1afc6a8cdcd43a118
I20260927 23:29:10.084827 5115 raft_consensus.cc:2454] T bd53bce1e3de426fbe05fd3225aae88d P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 1.
I20260927 23:29:10.104491 5866 ts_tablet_manager.cc:1939] T 1c89dab9709541c1afc6a8cdcd43a118 P f5ec98b2972d44838fa273eb7a10d01a: Deleting consensus metadata
I20260927 23:29:10.111968 4538 catalog_manager.cc:4841] TS f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): tablet 1c89dab9709541c1afc6a8cdcd43a118 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:29:10.112927 5866 tablet_replica.cc:330] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a: stopping tablet replica
I20260927 23:29:10.113760 5866 raft_consensus.cc:2229] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:10.114240 5866 raft_consensus.cc:2258] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:10.119877 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 707d73ec9c2b45d0a6391d053a0bca9a (DEFAULT_TABLE table=default.table_85c5c1d72c47496b94eb0f8c080ead12 [id=496c618bcbd04643b08992de376c904b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:10.120906 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707d73ec9c2b45d0a6391d053a0bca9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:10.122351 5866 ts_tablet_manager.cc:1905] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 23:29:10.123980 4548 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:29:10.128988 4951 tablet_service.cc:1467] Processing CreateTablet for tablet 707d73ec9c2b45d0a6391d053a0bca9a (DEFAULT_TABLE table=default.table_85c5c1d72c47496b94eb0f8c080ead12 [id=496c618bcbd04643b08992de376c904b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:10.130054 4951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707d73ec9c2b45d0a6391d053a0bca9a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260927 23:29:10.124228 4548 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
I20260927 23:29:10.132874 5098 tablet_service.cc:1467] Processing CreateTablet for tablet 707d73ec9c2b45d0a6391d053a0bca9a (DEFAULT_TABLE table=default.table_85c5c1d72c47496b94eb0f8c080ead12 [id=496c618bcbd04643b08992de376c904b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:10.133811 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 707d73ec9c2b45d0a6391d053a0bca9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:10.133850 5867 ts_tablet_manager.cc:1918] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:29:10.134187 5867 log.cc:1196] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/1c89dab9709541c1afc6a8cdcd43a118
I20260927 23:29:10.134868 5867 ts_tablet_manager.cc:1939] T 1c89dab9709541c1afc6a8cdcd43a118 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:10.138826 5869 tablet_bootstrap.cc:492] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:10.135382 4551 catalog_manager.cc:5495] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e62b4926a8448b2b5f4923553328bca (127.2.57.1). New cstate: current_term: 1 leader_uuid: "7e62b4926a8448b2b5f4923553328bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260927 23:29:10.150748 4538 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet 1c89dab9709541c1afc6a8cdcd43a118 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:29:10.159518 5867 tablet_replica.cc:330] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d: stopping tablet replica
I20260927 23:29:10.179003 5865 ts_tablet_manager.cc:1918] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:29:10.179471 5865 log.cc:1196] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/1c89dab9709541c1afc6a8cdcd43a118
I20260927 23:29:10.180131 5865 ts_tablet_manager.cc:1939] T 1c89dab9709541c1afc6a8cdcd43a118 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:10.182411 5865 tablet_replica.cc:330] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca: stopping tablet replica
I20260927 23:29:10.182605 4535 catalog_manager.cc:4841] TS 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): tablet 1c89dab9709541c1afc6a8cdcd43a118 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d]) successfully deleted
I20260927 23:29:10.183173 5865 raft_consensus.cc:2229] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: Raft consensus shutting down.
I20260927 23:29:10.184026 5865 raft_consensus.cc:2258] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:10.197242 5869 tablet_bootstrap.cc:654] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:10.201633 5867 raft_consensus.cc:2229] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 23:29:10.202308 5867 raft_consensus.cc:2258] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 23:29:10.208418 5867 ts_tablet_manager.cc:1905] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:10.232611 5865 ts_tablet_manager.cc:1905] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 23:29:10.251950 5867 ts_tablet_manager.cc:1918] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:29:10.252475 5867 log.cc:1196] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-1/wal/wals/b9f07e8a37404636bda765146b20fc60
I20260927 23:29:10.253242 5867 ts_tablet_manager.cc:1939] T b9f07e8a37404636bda765146b20fc60 P bad92a10de2c4459827cdbb18854d76d: Deleting consensus metadata
I20260927 23:29:10.255649 4627 catalog_manager.cc:2158] deleted orphaned HMS catalog entry for table [id=841dea283b0d475ebee6bc0637c75421]
I20260927 23:29:10.255807 4538 catalog_manager.cc:4841] TS bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863]) successfully deleted
I20260927 23:29:10.256549 4627 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:35136 (request call id 108) took 5662 ms. Trace:
I20260927 23:29:10.256822 4627 rpcz_store.cc:276] 0927 23:29:04.593755 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:04.593883 (+ 128us) service_pool.cc:227] Handling call
0927 23:29:07.798827 (+3204944us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:29:07.816201 (+ 17374us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:09.241392 (+1425191us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:09.242958 (+ 1566us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:09.242978 (+ 20us) write_op.cc:454] Released schema lock
0927 23:29:09.243438 (+ 460us) catalog_manager.cc:2155] Rolling back HMS table creation
0927 23:29:10.256489 (+1013051us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:09.242765 (+ 0us) op_driver.cc:372] HandleFailure(Illegal state: replica 83cd22a89b7846999c0e4dfab80ebe9d is not leader of this config: current role FOLLOWER)
0927 23:29:09.242797 (+ 32us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:09.242828 (+ 31us) write_op.cc:454] Released schema lock
0927 23:29:09.242848 (+ 20us) write_op.cc:335] FINISH: Op aborted
Metrics: {"Hive Metastore.queue_time_us":1865448,"Hive Metastore.run_cpu_time_us":926,"Hive Metastore.run_wall_time_us":570727,"child_traces":[["op",{}]]}
I20260927 23:29:10.257761 5869 tablet_bootstrap.cc:492] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:10.259307 4550 catalog_manager.cc:2148] created HMS catalog entry for table [id=ac5e07558ff04f98882932b135c35d7f]
I20260927 23:29:10.268114 5869 ts_tablet_manager.cc:1397] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.129s user 0.006s sys 0.004s
I20260927 23:29:10.271239 5869 raft_consensus.cc:348] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.280699 5869 raft_consensus.cc:374] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:10.281013 5869 raft_consensus.cc:729] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:10.280728 4548 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42536:
name: "default.Table_e2a4264ee2df4a97a99f0846c2489d0b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 23:29:10.285092 4738 catalog_manager.cc:2709] Deleting table default.table_68574d2015dd47bbb92137d783bad572 [id=572006748eda47c2bd6cdfd3142d5575] in response to Hive Metastore notification log event 26
W20260927 23:29:10.286046 4738 hms_notification_log_listener.cc:294] Failed to handle Hive Metastore notification: 26 DROP_TABLE default.table_68574d2015dd47bbb92137d783bad572: Not found: the table does not exist: table_id: "572006748eda47c2bd6cdfd3142d5575" table_name: "default.table_68574d2015dd47bbb92137d783bad572"
F20260927 23:29:10.286523 4738 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 28): 28 DROP_TABLE default.table_e2a4264ee2df4a97a99f0846c2489d0b
I20260927 23:29:10.286588 5869 consensus_queue.cc:260] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.288769 5869 ts_tablet_manager.cc:1428] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.020s user 0.006s sys 0.000s
I20260927 23:29:10.291356 5877 tablet_bootstrap.cc:492] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:10.287175 4550 rpcz_store.cc:275] Call kudu.master.MasterService.CreateTable from 127.0.0.1:42536 (request call id 36) took 1097 ms. Trace:
I20260927 23:29:10.292266 4550 rpcz_store.cc:276] 0927 23:29:09.189617 (+ 0us) service_pool.cc:170] Inserting onto call queue
0927 23:29:09.193085 (+ 3468us) service_pool.cc:227] Handling call
0927 23:29:09.693692 (+500607us) catalog_manager.cc:2075] Acquired catalog manager lock
0927 23:29:09.695340 (+ 1648us) catalog_manager.cc:2121] Created new table and tablet info
0927 23:29:10.259269 (+563929us) catalog_manager.cc:2147] Created new table in HMS catalog
0927 23:29:10.286541 (+ 27272us) catalog_manager.cc:2181] Wrote table and tablets to system table
0927 23:29:10.286868 (+ 327us) catalog_manager.cc:2215] Inserted table and tablets into CatalogManager maps
0927 23:29:10.287114 (+ 246us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0927 23:29:10.265956 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0927 23:29:10.266369 (+ 413us) write_op.cc:432] Acquiring schema lock in shared mode
0927 23:29:10.266394 (+ 25us) write_op.cc:435] Acquired schema lock
0927 23:29:10.266410 (+ 16us) tablet.cc:660] Decoding operations
0927 23:29:10.266708 (+ 298us) write_op.cc:620] Acquiring the partition lock for write op
0927 23:29:10.266749 (+ 41us) write_op.cc:641] Partition lock acquired for write op
0927 23:29:10.266775 (+ 26us) tablet.cc:683] Acquiring locks for 2 operations
0927 23:29:10.267005 (+ 230us) tablet.cc:699] Row locks acquired
0927 23:29:10.267022 (+ 17us) write_op.cc:260] PREPARE: finished
0927 23:29:10.267136 (+ 114us) write_op.cc:270] Start()
0927 23:29:10.267222 (+ 86us) write_op.cc:276] Timestamp: P: 1790551750267076 usec, L: 0
0927 23:29:10.267241 (+ 19us) op_driver.cc:331] REPLICATION: starting
0927 23:29:10.267753 (+ 512us) log.cc:842] Serialized 433 byte log entry
0927 23:29:10.283710 (+ 15957us) op_driver.cc:444] REPLICATION: finished
0927 23:29:10.284040 (+ 330us) write_op.cc:301] APPLY: starting
0927 23:29:10.284090 (+ 50us) tablet.cc:1365] starting BulkCheckPresence
0927 23:29:10.284191 (+ 101us) tablet.cc:1368] finished BulkCheckPresence
0927 23:29:10.284329 (+ 138us) tablet.cc:1370] starting ApplyRowOperation cycle
0927 23:29:10.284686 (+ 357us) tablet.cc:1381] finished ApplyRowOperation cycle
0927 23:29:10.284818 (+ 132us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0927 23:29:10.284856 (+ 38us) write_op.cc:312] APPLY: finished
0927 23:29:10.285037 (+ 181us) log.cc:842] Serialized 29 byte log entry
0927 23:29:10.285183 (+ 146us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:10.285248 (+ 65us) write_op.cc:454] Released schema lock
0927 23:29:10.285277 (+ 29us) write_op.cc:341] FINISH: Updating metrics
0927 23:29:10.285449 (+ 172us) write_op.cc:489] Releasing partition, row and schema locks
0927 23:29:10.285471 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"Hive Metastore.queue_time_us":292482,"Hive Metastore.run_cpu_time_us":566,"Hive Metastore.run_wall_time_us":270746,"child_traces":[["op",{"apply.queue_time_us":183,"apply.run_cpu_time_us":1572,"apply.run_wall_time_us":1571,"num_ops":2,"prepare.queue_time_us":5247,"prepare.run_cpu_time_us":2133,"prepare.run_wall_time_us":13450,"raft.queue_time_us":5757,"raft.run_cpu_time_us":660,"raft.run_wall_time_us":661,"replication_time_us":16390}]]}
*** Check failure stack trace: ***
I20260927 23:29:10.301155 5865 ts_tablet_manager.cc:1918] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:29:10.301609 5865 log.cc:1196] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-0/wal/wals/b9f07e8a37404636bda765146b20fc60
I20260927 23:29:10.302352 5865 ts_tablet_manager.cc:1939] T b9f07e8a37404636bda765146b20fc60 P 7e62b4926a8448b2b5f4923553328bca: Deleting consensus metadata
I20260927 23:29:10.307248 2276 master-stress-test.cc:580] Killed test thread 1/10
I20260927 23:29:10.307513 2276 master-stress-test.cc:578] Killing test thread 2/10
I20260927 23:29:10.309293 5877 tablet_bootstrap.cc:654] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:10.321667 5877 tablet_bootstrap.cc:492] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:10.322173 5877 ts_tablet_manager.cc:1397] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.031s user 0.007s sys 0.005s
I20260927 23:29:10.324640 5877 raft_consensus.cc:348] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.325426 5877 raft_consensus.cc:374] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:10.325732 5877 raft_consensus.cc:729] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:10.326460 5877 consensus_queue.cc:260] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.345534 5877 ts_tablet_manager.cc:1428] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.023s user 0.004s sys 0.001s
I20260927 23:29:10.442379 5866 ts_tablet_manager.cc:1918] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 23:29:10.443554 5911 tablet_bootstrap.cc:492] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
I20260927 23:29:10.445000 5866 log.cc:1196] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a: Deleting WAL directory at /tmp/dist-test-task6z6gmC/test-tmp/master-stress-test.0.CatalogManagerConfigurations_MasterStressTest.Test_3.1790551660889479-2276-0/minicluster-data/ts-2/wal/wals/b9f07e8a37404636bda765146b20fc60
I20260927 23:29:10.446770 5866 ts_tablet_manager.cc:1939] T b9f07e8a37404636bda765146b20fc60 P f5ec98b2972d44838fa273eb7a10d01a: Deleting consensus metadata
I20260927 23:29:10.457301 5864 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:10.460976 5864 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:29:10.463505 5864 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:29:10.465818 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 1766d227273e4c7592c4f0a20f2f7b17 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1766d227273e4c7592c4f0a20f2f7b17 consensus state has no leader
W20260927 23:29:10.472834 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 6: this message will repeat every 5th retry.
W20260927 23:29:10.477519 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:29:10.497493 5911 tablet_bootstrap.cc:654] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:10.522082 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
W20260927 23:29:10.522490 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
W20260927 23:29:10.522727 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
W20260927 23:29:10.523185 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
W20260927 23:29:10.523402 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
W20260927 23:29:10.523598 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
W20260927 23:29:10.523792 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
W20260927 23:29:10.523998 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
W20260927 23:29:10.524195 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:29:10.524405 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
@ 0x7fc06574b9a0 google::LogMessage::Flush() at ??:0
W20260927 23:29:10.541572 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:29:10.541977 5054 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:10.542708 5914 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:10.569041 5881 raft_consensus.cc:482] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:10.569505 5881 raft_consensus.cc:504] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.571347 5881 leader_election.cc:288] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:10.572381 4966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d73ec9c2b45d0a6391d053a0bca9a" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:10.572981 4966 raft_consensus.cc:2454] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 0.
I20260927 23:29:10.574146 4769 leader_election.cc:302] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:10.574843 5881 raft_consensus.cc:2781] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:10.575126 5881 raft_consensus.cc:482] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:10.576886 5881 raft_consensus.cc:3037] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
@ 0x7fc06574b70f google::LogMessage::~LogMessage() at ??:0
I20260927 23:29:10.579591 4810 tablet_service.cc:1467] Processing CreateTablet for tablet 7eaa190ae2764d468bd966c615b28652 (DEFAULT_TABLE table=default.table_1fbfdf99ff4e4165bfcf02ceb0247c87 [id=ac5e07558ff04f98882932b135c35d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:10.580524 4810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eaa190ae2764d468bd966c615b28652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:10.580379 4951 tablet_service.cc:1467] Processing CreateTablet for tablet 7eaa190ae2764d468bd966c615b28652 (DEFAULT_TABLE table=default.table_1fbfdf99ff4e4165bfcf02ceb0247c87 [id=ac5e07558ff04f98882932b135c35d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:10.581370 4951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eaa190ae2764d468bd966c615b28652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:10.583662 5881 raft_consensus.cc:504] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.587009 4966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d73ec9c2b45d0a6391d053a0bca9a" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d"
I20260927 23:29:10.587543 4966 raft_consensus.cc:3037] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:10.594743 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d73ec9c2b45d0a6391d053a0bca9a" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:10.598562 4769 leader_election.cc:341] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:10.597162 4966 raft_consensus.cc:2454] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e62b4926a8448b2b5f4923553328bca in term 1.
I20260927 23:29:10.601161 4769 leader_election.cc:302] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:10.601212 5098 tablet_service.cc:1467] Processing CreateTablet for tablet 7eaa190ae2764d468bd966c615b28652 (DEFAULT_TABLE table=default.table_1fbfdf99ff4e4165bfcf02ceb0247c87 [id=ac5e07558ff04f98882932b135c35d7f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 23:29:10.602334 5098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7eaa190ae2764d468bd966c615b28652. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 23:29:10.607549 5869 tablet_bootstrap.cc:492] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca: Bootstrap starting.
I20260927 23:29:10.610061 5881 leader_election.cc:288] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Requested vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:10.610569 5881 raft_consensus.cc:2781] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:10.610929 5881 raft_consensus.cc:686] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [term 1 LEADER]: Becoming Leader. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Running, Role: LEADER
I20260927 23:29:10.611613 5881 consensus_queue.cc:237] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.613401 5869 tablet_bootstrap.cc:654] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca: Neither blocks nor log segments found. Creating new log.
W20260927 23:29:10.614262 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
I20260927 23:29:10.631162 5869 tablet_bootstrap.cc:492] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca: No bootstrap required, opened a new log
I20260927 23:29:10.631752 5869 ts_tablet_manager.cc:1397] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.000s
I20260927 23:29:10.634282 5869 raft_consensus.cc:348] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.645906 5869 raft_consensus.cc:374] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:10.648401 5869 raft_consensus.cc:729] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e62b4926a8448b2b5f4923553328bca, State: Initialized, Role: FOLLOWER
I20260927 23:29:10.649225 5869 consensus_queue.cc:260] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.652213 5869 ts_tablet_manager.cc:1428] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca: Time spent starting tablet: real 0.020s user 0.005s sys 0.000s
I20260927 23:29:10.653388 5877 tablet_bootstrap.cc:492] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d: Bootstrap starting.
I20260927 23:29:10.659073 5877 tablet_bootstrap.cc:654] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d: Neither blocks nor log segments found. Creating new log.
I20260927 23:29:10.669853 5918 raft_consensus.cc:482] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:10.670329 5918 raft_consensus.cc:504] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:10.671273 4769 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.2:45717, user_credentials={real_user=slave}} blocked reactor thread for 70186.3us
I20260927 23:29:10.672086 5918 leader_election.cc:288] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:10.673238 4972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eaa190ae2764d468bd966c615b28652" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:10.676470 5114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d73ec9c2b45d0a6391d053a0bca9a" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
W20260927 23:29:10.678679 4769 leader_election.cc:341] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:10.680033 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eaa190ae2764d468bd966c615b28652" candidate_uuid: "7e62b4926a8448b2b5f4923553328bca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
W20260927 23:29:10.684584 4769 leader_election.cc:341] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:10.687304 5911 tablet_bootstrap.cc:492] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
@ 0x7fc0707e6aea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
W20260927 23:29:10.691417 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
W20260927 23:29:10.691646 4551 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
I20260927 23:29:10.699918 5877 tablet_bootstrap.cc:492] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d: No bootstrap required, opened a new log
I20260927 23:29:10.700361 5877 ts_tablet_manager.cc:1397] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d: Time spent bootstrapping tablet: real 0.047s user 0.003s sys 0.010s
W20260927 23:29:10.704576 4769 leader_election.cc:341] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519): Illegal state: must be running to vote when last-logged opid is not known
I20260927 23:29:10.704945 4769 leader_election.cc:302] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca; no voters: bad92a10de2c4459827cdbb18854d76d, f5ec98b2972d44838fa273eb7a10d01a
I20260927 23:29:10.706171 5877 raft_consensus.cc:348] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.706892 5877 raft_consensus.cc:374] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:10.707132 5877 raft_consensus.cc:729] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Initialized, Role: FOLLOWER
I20260927 23:29:10.707839 5877 consensus_queue.cc:260] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.733701 5918 raft_consensus.cc:2726] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:10.733429 5911 ts_tablet_manager.cc:1397] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.290s user 0.004s sys 0.026s
I20260927 23:29:10.747665 4551 catalog_manager.cc:5495] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e62b4926a8448b2b5f4923553328bca (127.2.57.1). New cstate: current_term: 1 leader_uuid: "7e62b4926a8448b2b5f4923553328bca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } health_report { overall_health: UNKNOWN } } }
I20260927 23:29:10.753032 5877 ts_tablet_manager.cc:1428] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d: Time spent starting tablet: real 0.052s user 0.005s sys 0.000s
@ 0x7fc0707e633c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
W20260927 23:29:10.773789 4550 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:29:10.774051 4550 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
I20260927 23:29:10.783032 5911 raft_consensus.cc:348] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.783757 5911 raft_consensus.cc:374] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:10.784013 5911 raft_consensus.cc:729] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
@ 0x7fc0707e9fd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
I20260927 23:29:10.802469 5911 consensus_queue.cc:260] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.822600 5911 ts_tablet_manager.cc:1428] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.068s user 0.004s sys 0.001s
@ 0x7fc0707e9f8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
I20260927 23:29:10.832441 5911 tablet_bootstrap.cc:492] T 7eaa190ae2764d468bd966c615b28652 P f5ec98b2972d44838fa273eb7a10d01a: Bootstrap starting.
W20260927 23:29:10.841113 4552 catalog_manager.cc:5328] Ignoring report from unknown tablet df6ea36da9f6455bb399c2229dcb3790
W20260927 23:29:10.841374 4552 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
W20260927 23:29:10.841533 4552 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
W20260927 23:29:10.841681 4552 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:29:10.841821 4552 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
@ 0x7fc0707e9f1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
W20260927 23:29:10.856094 4552 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
@ 0x7fc0707e9ee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
I20260927 23:29:10.873342 5911 tablet_bootstrap.cc:654] T 7eaa190ae2764d468bd966c615b28652 P f5ec98b2972d44838fa273eb7a10d01a: Neither blocks nor log segments found. Creating new log.
@ 0x7fc0707e91de std::__1::__function::__func<>::operator()() at ??:0
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
W20260927 23:29:10.901300 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 6: this message will repeat every 5th retry.
@ 0x5805ba std::__1::function<>::operator()() at ??:?
W20260927 23:29:10.913267 4552 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:29:10.929350 5911 tablet_bootstrap.cc:492] T 7eaa190ae2764d468bd966c615b28652 P f5ec98b2972d44838fa273eb7a10d01a: No bootstrap required, opened a new log
I20260927 23:29:10.929790 5911 ts_tablet_manager.cc:1397] T 7eaa190ae2764d468bd966c615b28652 P f5ec98b2972d44838fa273eb7a10d01a: Time spent bootstrapping tablet: real 0.098s user 0.012s sys 0.000s
I20260927 23:29:10.932823 5911 raft_consensus.cc:348] T 7eaa190ae2764d468bd966c615b28652 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.933516 5911 raft_consensus.cc:374] T 7eaa190ae2764d468bd966c615b28652 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 23:29:10.933765 5911 raft_consensus.cc:729] T 7eaa190ae2764d468bd966c615b28652 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ec98b2972d44838fa273eb7a10d01a, State: Initialized, Role: FOLLOWER
@ 0x7fc0678fcdc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
I20260927 23:29:10.945355 5911 consensus_queue.cc:260] T 7eaa190ae2764d468bd966c615b28652 P f5ec98b2972d44838fa273eb7a10d01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
@ 0x7fc068f6a6db start_thread at ??:0
@ 0x7fc0643f061f clone at ??:0
I20260927 23:29:10.973634 5911 ts_tablet_manager.cc:1428] T 7eaa190ae2764d468bd966c615b28652 P f5ec98b2972d44838fa273eb7a10d01a: Time spent starting tablet: real 0.044s user 0.006s sys 0.002s
I20260927 23:29:10.975117 5909 raft_consensus.cc:482] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:10.975679 5909 raft_consensus.cc:504] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:10.985780 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eaa190ae2764d468bd966c615b28652" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:10.986471 4830 raft_consensus.cc:2454] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:10.995265 5909 leader_election.cc:288] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:10.998785 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eaa190ae2764d468bd966c615b28652" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a" is_pre_election: true
I20260927 23:29:10.999369 5118 raft_consensus.cc:2454] T 7eaa190ae2764d468bd966c615b28652 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 0.
I20260927 23:29:10.999336 4907 leader_election.cc:302] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:11.019887 5909 raft_consensus.cc:2781] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 23:29:11.040366 5909 raft_consensus.cc:482] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 23:29:11.040755 5909 raft_consensus.cc:3037] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:11.059731 5918 consensus_queue.cc:1035] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:29:11.070297 4874 debug-util.cc:398] Leaking SignalData structure 0x7b08000fbde0 after lost signal to thread 4744
W20260927 23:29:11.071439 4874 debug-util.cc:398] Leaking SignalData structure 0x7b0800100fc0 after lost signal to thread 4877
I20260927 23:29:11.088253 5909 raft_consensus.cc:504] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:11.090797 5909 leader_election.cc:288] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Requested vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), f5ec98b2972d44838fa273eb7a10d01a (127.2.57.3:45519)
I20260927 23:29:11.090921 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eaa190ae2764d468bd966c615b28652" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca"
I20260927 23:29:11.091423 4830 raft_consensus.cc:3037] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:11.096707 5118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7eaa190ae2764d468bd966c615b28652" candidate_uuid: "bad92a10de2c4459827cdbb18854d76d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5ec98b2972d44838fa273eb7a10d01a"
I20260927 23:29:11.097266 5118 raft_consensus.cc:3037] T 7eaa190ae2764d468bd966c615b28652 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:11.119860 4830 raft_consensus.cc:2454] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:11.121057 4907 leader_election.cc:302] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d; no voters:
I20260927 23:29:11.125434 5118 raft_consensus.cc:2454] T 7eaa190ae2764d468bd966c615b28652 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bad92a10de2c4459827cdbb18854d76d in term 1.
I20260927 23:29:11.127430 5909 raft_consensus.cc:2781] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 23:29:11.127946 5909 raft_consensus.cc:686] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [term 1 LEADER]: Becoming Leader. State: Replica: bad92a10de2c4459827cdbb18854d76d, State: Running, Role: LEADER
I20260927 23:29:11.132680 5906 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:11.133111 5906 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:11.134896 5906 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:11.144739 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:29:11.170886 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 6: this message will repeat every 5th retry.
W20260927 23:29:11.175379 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:29:11.175750 5051 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:11.176829 5906 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:11.184468 5937 consensus_queue.cc:1035] T bd53bce1e3de426fbe05fd3225aae88d P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 23:29:11.188267 5914 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:11.188697 5914 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:11.189342 5909 consensus_queue.cc:237] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:11.190548 5914 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:11.192517 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:29:11.194610 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:29:11.194947 5051 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:11.196002 5906 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:11.196949 5906 raft_consensus.cc:482] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:11.197361 5906 raft_consensus.cc:504] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:11.199793 5906 leader_election.cc:288] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:11.202049 4966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d73ec9c2b45d0a6391d053a0bca9a" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bad92a10de2c4459827cdbb18854d76d" is_pre_election: true
I20260927 23:29:11.202910 4966 raft_consensus.cc:2379] T 707d73ec9c2b45d0a6391d053a0bca9a P bad92a10de2c4459827cdbb18854d76d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5ec98b2972d44838fa273eb7a10d01a in current term 1: Already voted for candidate 7e62b4926a8448b2b5f4923553328bca in this term.
I20260927 23:29:11.205016 4830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "707d73ec9c2b45d0a6391d053a0bca9a" candidate_uuid: "f5ec98b2972d44838fa273eb7a10d01a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e62b4926a8448b2b5f4923553328bca" is_pre_election: true
I20260927 23:29:11.206566 5051 leader_election.cc:302] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:11.207412 5914 raft_consensus.cc:3037] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 23:29:11.214468 5914 raft_consensus.cc:2726] T 707d73ec9c2b45d0a6391d053a0bca9a P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:11.256057 5906 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:11.256521 5906 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:11.259686 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:29:11.263783 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:29:11.264115 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:11.264817 5914 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:11.265480 5906 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:11.312945 5941 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f8460f73b40c41a28ec020fc04203564)
I20260927 23:29:11.313395 5941 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:11.315462 5941 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:29:11.318819 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 3 candidate_status { last_received { term: 2 index: 166 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" is_pre_election: true
I20260927 23:29:11.319404 4640 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9ed71d97661411891f2d2de4cdc2269 in term 2.
I20260927 23:29:11.320564 5474 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d, d9ed71d97661411891f2d2de4cdc2269; no voters:
I20260927 23:29:11.321259 5941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260927 23:29:11.321565 5941 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f8460f73b40c41a28ec020fc04203564)
I20260927 23:29:11.321890 5941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 2 FOLLOWER]: Advancing to term 3
I20260927 23:29:11.327769 5941 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:11.330149 5941 leader_election.cc:288] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 3 election: Requested vote from peers f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527), 83cd22a89b7846999c0e4dfab80ebe9d (127.2.57.60:33749)
I20260927 23:29:11.331079 4640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d9ed71d97661411891f2d2de4cdc2269" candidate_term: 3 candidate_status { last_received { term: 2 index: 166 } } ignore_live_leader: false dest_uuid: "83cd22a89b7846999c0e4dfab80ebe9d"
I20260927 23:29:11.331341 5942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f8460f73b40c41a28ec020fc04203564)
I20260927 23:29:11.331904 5942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:11.333081 4640 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 2 FOLLOWER]: Advancing to term 3
I20260927 23:29:11.334408 5942 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
I20260927 23:29:11.334815 5544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 166 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269" is_pre_election: true
I20260927 23:29:11.335511 5544 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83cd22a89b7846999c0e4dfab80ebe9d in current term 3: Already voted for candidate d9ed71d97661411891f2d2de4cdc2269 in this term.
I20260927 23:29:11.340190 4640 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9ed71d97661411891f2d2de4cdc2269 in term 3.
I20260927 23:29:11.341897 5474 leader_election.cc:302] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d, d9ed71d97661411891f2d2de4cdc2269; no voters:
I20260927 23:29:11.342634 5941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 3 FOLLOWER]: Leader election won for term 3
I20260927 23:29:11.358418 5906 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:11.358907 5906 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:11.360965 5906 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:11.366027 5941 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [term 3 LEADER]: Becoming Leader. State: Replica: d9ed71d97661411891f2d2de4cdc2269, State: Running, Role: LEADER
I20260927 23:29:11.366921 5941 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 166, Committed index: 166, Last appended: 2.166, Last appended by leader: 166, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
W20260927 23:29:11.371157 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:11.371018 5943 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9ed71d97661411891f2d2de4cdc2269. Latest consensus state: current_term: 3 leader_uuid: "d9ed71d97661411891f2d2de4cdc2269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:11.371863 5943 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: This master's current role is: LEADER
W20260927 23:29:11.372983 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:11.373301 5051 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:11.375456 5914 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:11.433827 5944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 23:29:11.459831 5944 catalog_manager.cc:669] Loaded metadata for table default.table_8a2ddb6dcbbc4b8bba970c3bf5c37848 [id=078228a392424e6c96b3e5dbc1a7380d]
I20260927 23:29:11.482033 5944 catalog_manager.cc:669] Loaded metadata for table default.table_1c3db2bbcc0e488ea4b6637492ff4221 [id=09f1820cca0c454e928a02f0677e7c9d]
I20260927 23:29:11.510859 5944 catalog_manager.cc:669] Loaded metadata for table default.table_85c5c1d72c47496b94eb0f8c080ead12 [id=496c618bcbd04643b08992de376c904b]
I20260927 23:29:11.512562 5944 catalog_manager.cc:669] Loaded metadata for table default.table_829e227cbcac4bc585663258666cd43a [id=70012e8fafc049d2833983e702dd05ba]
I20260927 23:29:11.514127 5944 catalog_manager.cc:669] Loaded metadata for table default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc]
I20260927 23:29:11.515619 5944 catalog_manager.cc:669] Loaded metadata for table default.table_1fbfdf99ff4e4165bfcf02ceb0247c87 [id=ac5e07558ff04f98882932b135c35d7f]
I20260927 23:29:11.517182 5944 catalog_manager.cc:669] Loaded metadata for table default.table_ecd08889b3cb4f09b30c4588cdceda4b [id=aea1156e91a84d018912c856e3f89d34]
I20260927 23:29:11.518625 5944 catalog_manager.cc:669] Loaded metadata for table default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40]
I20260927 23:29:11.520094 5944 catalog_manager.cc:669] Loaded metadata for table default.table_143d2fa1ac844e81b1392bd3cc4d95c7 [id=fa9d9f99dc5d4747a22f79e6a1533674]
I20260927 23:29:11.575457 5937 consensus_queue.cc:1035] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:29:11.590149 5175 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.57.61:46527)
I20260927 23:29:11.604655 5918 consensus_queue.cc:1035] T 707d73ec9c2b45d0a6391d053a0bca9a P 7e62b4926a8448b2b5f4923553328bca [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 23:29:11.609177 5052 connection.cc:535] client connection to 127.2.57.61:46527 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:29:11.610083 4908 connection.cc:535] client connection to 127.2.57.61:46527 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:29:11.632586 4767 connection.cc:535] client connection to 127.2.57.61:46527 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:29:11.641983 4615 connection.cc:535] client connection to 127.2.57.61:46527 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:29:11.642354 4615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107) [suppressed 9 similar messages]
W20260927 23:29:11.652765 5471 connection.cc:535] client connection to 127.2.57.61:46527 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:29:11.653286 5471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:29:11.657469 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:11.688262 5944 tablet_loader.cc:96] loaded metadata for tablet 1572bbcfebce4d218b1a0f38ecab0384 (table default.table_ecd08889b3cb4f09b30c4588cdceda4b [id=aea1156e91a84d018912c856e3f89d34])
I20260927 23:29:11.690460 5944 tablet_loader.cc:96] loaded metadata for tablet 340a88a1702d48be9a8373fb3c6d540c (table default.table_829e227cbcac4bc585663258666cd43a [id=70012e8fafc049d2833983e702dd05ba])
I20260927 23:29:11.692394 5944 tablet_loader.cc:96] loaded metadata for tablet 707d73ec9c2b45d0a6391d053a0bca9a (table default.table_85c5c1d72c47496b94eb0f8c080ead12 [id=496c618bcbd04643b08992de376c904b])
W20260927 23:29:11.692981 5952 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:29:11.612586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.57.61:46527 (local address 127.2.57.2:56907)
0927 23:29:11.634612 (+ 22026us) negotiation.cc:107] Waiting for socket to connect
0927 23:29:11.645582 (+ 10970us) client_negotiation.cc:174] Beginning negotiation
0927 23:29:11.646961 (+ 1379us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 23:29:11.664500 (+ 17539us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":21574,"thread_start_us":24237,"threads_started":1}
I20260927 23:29:11.694437 5944 tablet_loader.cc:96] loaded metadata for tablet 7eaa190ae2764d468bd966c615b28652 (table default.table_1fbfdf99ff4e4165bfcf02ceb0247c87 [id=ac5e07558ff04f98882932b135c35d7f])
W20260927 23:29:11.695809 4615 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.61:46527, user_credentials={real_user=slave}} blocked reactor thread for 53358.5us
W20260927 23:29:11.697069 5471 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.2.57.61:46527, user_credentials={real_user=slave}} blocked reactor thread for 43844.6us
I20260927 23:29:11.701247 5944 tablet_loader.cc:96] loaded metadata for tablet 8546d6e754784ff886f421eb0ddb796a (table default.table_1c3db2bbcc0e488ea4b6637492ff4221 [id=09f1820cca0c454e928a02f0677e7c9d])
W20260927 23:29:11.705176 5953 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:29:11.618263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.57.61:46527 (local address 127.2.57.3:46305)
0927 23:29:11.704051 (+ 85788us) negotiation.cc:107] Waiting for socket to connect
0927 23:29:11.704336 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":84863,"thread_start_us":84437,"threads_started":1}
W20260927 23:29:11.706578 5954 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:29:11.634694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.57.61:46527 (local address 127.2.57.1:60735)
0927 23:29:11.697629 (+ 62935us) negotiation.cc:107] Waiting for socket to connect
0927 23:29:11.697888 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":62472,"thread_start_us":60729,"threads_started":1}
W20260927 23:29:11.714918 5950 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 23:29:11.598168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.57.61:46527 (local address 127.0.0.1:33634)
0927 23:29:11.699847 (+101679us) negotiation.cc:107] Waiting for socket to connect
0927 23:29:11.700207 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":101180,"threads_started":1}
I20260927 23:29:11.716959 5944 tablet_loader.cc:96] loaded metadata for tablet 935c9a2e4d6c416db02443d2aaa57752 (table default.table_0ec8d1f3b75e4c44b6b60deb61245f76 [id=f7cae8537adf4c2bb9ae7a8c9c77ad40])
W20260927 23:29:11.717072 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:29:11.717671 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:11.718282 5944 tablet_loader.cc:96] loaded metadata for tablet bd53bce1e3de426fbe05fd3225aae88d (table default.table_99f8c479cf8848c08b0136e79c68b658 [id=983540517b0b4405b79b922358e1c3bc])
I20260927 23:29:11.718422 5942 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260927 23:29:11.719478 5944 tablet_loader.cc:96] loaded metadata for tablet be4adc02302b44c2b20a2be202ebe68d (table default.table_8a2ddb6dcbbc4b8bba970c3bf5c37848 [id=078228a392424e6c96b3e5dbc1a7380d])
I20260927 23:29:11.720722 5944 tablet_loader.cc:96] loaded metadata for tablet fe9cb9be216d42fa81948f5d9da3d4d8 (table default.table_143d2fa1ac844e81b1392bd3cc4d95c7 [id=fa9d9f99dc5d4747a22f79e6a1533674])
I20260927 23:29:11.724053 5944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 23:29:11.740934 5944 catalog_manager.cc:1259] Loaded cluster ID: 6dc11a7b873d4b11830ae46041eb6de8
I20260927 23:29:11.741214 5944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260927 23:29:11.743999 5471 leader_election.cc:334] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:29:11.746675 5471 leader_election.cc:334] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:29:11.778971 5944 catalog_manager.cc:1504] Loading token signing keys...
W20260927 23:29:11.782438 5471 consensus_peers.cc:487] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 -> Peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Couldn't send request to peer f8460f73b40c41a28ec020fc04203564. Status: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 23:29:11.801371 5944 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269: Loaded TSK: 0
I20260927 23:29:11.802843 5944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 23:29:11.837503 5944 catalog_manager.cc:1527] Loading latest processed Hive Metastore notification log event ID...
I20260927 23:29:11.872159 5944 catalog_manager.cc:5717] Last processed Hive Metastore notification event ID: 25
W20260927 23:29:11.888507 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
I20260927 23:29:11.898677 5165 heartbeater.cc:503] Master 127.2.57.62:35155 was elected leader, sending a full tablet report...
I20260927 23:29:11.957289 4640 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Refusing update from remote peer d9ed71d97661411891f2d2de4cdc2269: Log matching property violated. Preceding OpId in replica: term: 2 index: 166. Preceding OpId from leader: term: 3 index: 167. (index mismatch)
I20260927 23:29:11.958823 5943 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 167, Last known committed idx: 166, Time since last communication: 0.000s
W20260927 23:29:11.968802 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
W20260927 23:29:11.970517 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:29:11.971437 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet b373bde60f734d049c4a45255e2b2e2a
W20260927 23:29:11.972338 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
W20260927 23:29:11.972920 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet c739222645654eb39285a2ad1b6c35f5
W20260927 23:29:11.973433 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:29:11.975747 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 309465f419c24549bf5416a4503b9299
W20260927 23:29:11.976218 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet df6ea36da9f6455bb399c2229dcb3790
W20260927 23:29:11.984315 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:29:11.984884 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:29:11.985319 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
W20260927 23:29:11.985746 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
W20260927 23:29:11.986182 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
W20260927 23:29:11.990800 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
W20260927 23:29:11.991945 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
W20260927 23:29:11.992439 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
W20260927 23:29:11.995875 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:29:11.996393 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
W20260927 23:29:11.997164 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
W20260927 23:29:11.999725 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet b4ec12a496904c3599bdc001476ebee9
W20260927 23:29:12.000147 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet f171cc1f0094480a9cc6357eab93dff1
W20260927 23:29:12.000403 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
W20260927 23:29:12.001082 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
W20260927 23:29:12.001394 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:29:12.001591 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
I20260927 23:29:12.004685 5942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9ed71d97661411891f2d2de4cdc2269. Latest consensus state: current_term: 3 leader_uuid: "d9ed71d97661411891f2d2de4cdc2269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:12.005342 5942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: This master's current role is: FOLLOWER
W20260927 23:29:12.002038 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
W20260927 23:29:12.041883 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:12.042335 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
W20260927 23:29:12.042580 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:29:12.042222 5943 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "d9ed71d97661411891f2d2de4cdc2269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:12.043007 5943 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9ed71d97661411891f2d2de4cdc2269 [sys.catalog]: This master's current role is: LEADER
W20260927 23:29:12.042796 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
W20260927 23:29:12.043604 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet e56378a5d42e47d395c881a0898da495
W20260927 23:29:12.044055 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
W20260927 23:29:12.044804 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet e4e5cb0d645846159e09ab71a897f6df
W20260927 23:29:12.045068 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:29:12.045287 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:29:12.045490 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
W20260927 23:29:12.045709 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:29:12.045912 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
W20260927 23:29:12.046104 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet c5552715138046279f00dcf497d05a72
W20260927 23:29:12.046311 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:29:12.046527 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
W20260927 23:29:12.046749 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
W20260927 23:29:12.046969 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 9912e4dd70ba482398fbd831a8242f12
W20260927 23:29:12.047197 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:29:11.891204 5504 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34164:
name: "default.Table_e2a4264ee2df4a97a99f0846c2489d0b"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:29:12.047425 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:29:12.047596 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:29:12.047752 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:29:12.047897 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:29:12.048164 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
W20260927 23:29:12.048398 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:29:12.059144 5661 catalog_manager.cc:2709] Deleting table default.table_68574d2015dd47bbb92137d783bad572 [id=572006748eda47c2bd6cdfd3142d5575] in response to Hive Metastore notification log event 26
I20260927 23:29:12.059651 5942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "d9ed71d97661411891f2d2de4cdc2269" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } } }
I20260927 23:29:12.060374 5942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [sys.catalog]: This master's current role is: FOLLOWER
W20260927 23:29:12.063694 5661 hms_notification_log_listener.cc:294] Failed to handle Hive Metastore notification: 26 DROP_TABLE default.table_68574d2015dd47bbb92137d783bad572: Not found: the table does not exist: table_id: "572006748eda47c2bd6cdfd3142d5575" table_name: "default.table_68574d2015dd47bbb92137d783bad572"
F20260927 23:29:12.064087 5661 hms_notification_log_listener.cc:264] Received out-of-order notification log event (last processed event ID: 28): 28 DROP_TABLE default.table_e2a4264ee2df4a97a99f0846c2489d0b
*** Check failure stack trace: ***
I20260927 23:29:11.891275 5499 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34164:
name: "default.Table_68574d2015dd47bbb92137d783bad572"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 23:29:12.077946 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:29:12.078281 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
I20260927 23:29:12.104015 5932 consensus_queue.cc:1035] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
@ 0x7ffb4e27c9a0 google::LogMessage::Flush() at ??:0
W20260927 23:29:12.139613 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
I20260927 23:29:12.190881 5496 catalog_manager.cc:5495] T 7eaa190ae2764d468bd966c615b28652 P 7e62b4926a8448b2b5f4923553328bca reported cstate change: term changed from 0 to 1, leader changed from <none> to bad92a10de2c4459827cdbb18854d76d (127.2.57.2). New cstate: current_term: 1 leader_uuid: "bad92a10de2c4459827cdbb18854d76d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } }
@ 0x7ffb4e27c70f google::LogMessage::~LogMessage() at ??:0
I20260927 23:29:12.234145 5976 consensus_queue.cc:1035] T 7eaa190ae2764d468bd966c615b28652 P bad92a10de2c4459827cdbb18854d76d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260927 23:29:12.253641 5527 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:12.254060 5527 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:12.255717 5527 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:12.258301 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:29:12.260149 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:12.260532 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:12.261471 5906 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:29:12.274156 4880 heartbeater.cc:503] Master 127.2.57.62:35155 was elected leader, sending a full tablet report...
W20260927 23:29:12.276187 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:12.282271 5906 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:12.282716 5906 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:12.283998 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
W20260927 23:29:12.284583 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:12.284893 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
W20260927 23:29:12.285079 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
W20260927 23:29:12.285238 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
W20260927 23:29:12.285507 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
W20260927 23:29:12.286077 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:29:12.286839 5906 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:12.288156 5021 heartbeater.cc:503] Master 127.2.57.62:35155 was elected leader, sending a full tablet report...
W20260927 23:29:12.288818 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:29:12.289189 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:12.290194 5906 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
@ 0x7ffb59317aea kudu::master::HmsNotificationLogListenerTask::Poll() at ??:0
W20260927 23:29:12.330581 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 289d65ad2eb64788a6fa7dbc1b48122c
W20260927 23:29:12.336512 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet df6ea36da9f6455bb399c2229dcb3790
W20260927 23:29:12.336825 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
W20260927 23:29:12.337073 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 48dcf44bb7b744b9a98626dbfb6f4e6c
W20260927 23:29:12.337445 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 9dffe7e40ecb40c68a06f0cf43861777
W20260927 23:29:12.337697 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 0d90eb9269884933a9af575b5af8a47e
W20260927 23:29:12.338068 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet bdecd389ff0341f4ab867eb160786ea9
W20260927 23:29:12.338300 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
W20260927 23:29:12.338515 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
W20260927 23:29:12.338742 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c7f48abe7db4a5aa39892af9a3e3713
W20260927 23:29:12.338970 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
W20260927 23:29:12.339190 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
W20260927 23:29:12.339412 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
W20260927 23:29:12.339624 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 360070544fca4f62b0aac16d5889e1bc
W20260927 23:29:12.344871 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 2dcb78350f4c4b73beada474bcb01169
W20260927 23:29:12.345168 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 260ab7817da1431ebd80ee7dd00c40b4
W20260927 23:29:12.345402 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
W20260927 23:29:12.345641 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 43f8e7644db24be5a758b437a5f471c0
W20260927 23:29:12.345973 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
W20260927 23:29:12.346457 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
W20260927 23:29:12.347992 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet ac88a4e4bee64586bef7b6b409a63f4c
W20260927 23:29:12.348261 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
W20260927 23:29:12.348508 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
W20260927 23:29:12.350303 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:29:12.350559 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
W20260927 23:29:12.350807 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
W20260927 23:29:12.351035 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
W20260927 23:29:12.351426 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 2826716e5b184cc2b60d556ca2b89f4d
W20260927 23:29:12.353504 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 7a2477e4ce30497fa94f2dfdb3c31a3d
W20260927 23:29:12.358131 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:29:12.361567 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:29:12.372185 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 6c3c55f368dd4ba991daded55d2e3069
W20260927 23:29:12.387101 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet 3a0c6ad197164aa98d73d357de26b488
W20260927 23:29:12.415853 5496 catalog_manager.cc:5328] Ignoring report from unknown tablet a483b5e32d55406bb3342af93f755028
@ 0x7ffb5931733c kudu::master::HmsNotificationLogListenerTask::RunLoop() at ??:0
W20260927 23:29:12.438974 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet e7a73bf1fc35439ea3e39ffd843ebfb1
I20260927 23:29:12.454121 5804 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:12.454541 5804 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:12.456318 5804 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:12.456725 5804 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:12.457072 5804 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:12.458751 5804 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:12.462128 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:29:12.462706 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:29:12.466370 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 38cd408c3dc34fabbab5906286e7580a
W20260927 23:29:12.468549 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:29:12.468914 5051 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
W20260927 23:29:12.469825 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:29:12.470157 5051 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:12.470719 5804 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:12.470988 5527 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:12.498862 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 6: this message will repeat every 5th retry.
W20260927 23:29:12.502974 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet cc208e8764244ac68b2a73be1b3d957e
W20260927 23:29:12.521951 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 70cb3313148a47f8b6c692222508eb88
@ 0x7ffb5931afd2 kudu::master::HmsNotificationLogListenerTask::Init()::$_0::operator()() at ??:0
@ 0x7ffb5931af8a _ZNSt3__18__invokeIRZN4kudu6master30HmsNotificationLogListenerTask4InitEvE3$_0JEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOS6_DpOS7_ at ??:0
W20260927 23:29:12.566066 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 0c8328db65eb43ebb79cd9d7a1ddb0d8
W20260927 23:29:12.582811 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 77734e994d5d40dfb3efca344d396956
W20260927 23:29:12.608933 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 64cfbad1561a41048009b5fc819cc5de
@ 0x7ffb5931af1a std::__1::__invoke_void_return_wrapper<>::__call<>() at ??:0
W20260927 23:29:12.633414 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet c4deca666d4e4d25aae3ed6f13e97c4d
@ 0x7ffb5931aee2 std::__1::__function::__alloc_func<>::operator()() at ??:0
W20260927 23:29:12.695215 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet bbdfdfc23b0c4e3d9224ac2815c6cc65
W20260927 23:29:12.695508 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 6846550519ce44fdafbc895ddca98d8b
@ 0x7ffb5931a1de std::__1::__function::__func<>::operator()() at ??:0
@ 0x580785 std::__1::__function::__value_func<>::operator()() at ??:?
W20260927 23:29:12.741452 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 5073ab2faa0042569b11193952cce2ac
W20260927 23:29:12.741721 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 0a3148da66024c4684d5eaf3d60c8c58
W20260927 23:29:12.741920 5522 catalog_manager.cc:5328] Ignoring report from unknown tablet 667f1147ecae4874854935982dd19e96
@ 0x5805ba std::__1::function<>::operator()() at ??:?
I20260927 23:29:12.759603 5989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:29:12.761796 5989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
@ 0x7ffb5042ddc7 kudu::Thread::SuperviseThread() at ??:0
@ 0x4230f0 __tsan_thread_start_func at ??:?
I20260927 23:29:12.770355 5544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" candidate_term: 4 candidate_status { last_received { term: 3 index: 168 } } ignore_live_leader: false dest_uuid: "d9ed71d97661411891f2d2de4cdc2269" is_pre_election: true
I20260927 23:29:12.778453 5989 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
@ 0x7ffb51a9b6db start_thread at ??:0
@ 0x7ffb4cf2161f clone at ??:0
W20260927 23:29:12.821322 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:29:12.821789 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:12.822552 5989 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:29:13.191706 5176 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.57.62:35155)
W20260927 23:29:13.192906 5053 connection.cc:535] client connection to 127.2.57.62:35155 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 23:29:13.193696 4909 connection.cc:535] client connection to 127.2.57.62:35155 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 23:29:13.348040 6002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:29:13.348770 6002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:13.360654 5527 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:13.361152 5527 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:13.362984 5527 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:13.366753 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:29:13.367599 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:13.367990 5051 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:13.368803 5804 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:13.387764 6002 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:29:13.414178 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:29:13.415547 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:29:13.416057 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:13.416884 6002 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:29:13.462352 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9f07e8a37404636bda765146b20fc60 consensus state has no leader
I20260927 23:29:13.469521 5527 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:13.470034 5527 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:13.471866 5527 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:13.474030 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:29:13.477456 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:29:13.477862 5051 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:13.478850 5527 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:13.583134 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:13.707381 5527 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:13.707945 5527 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:13.709930 5527 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:13.711937 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:29:13.712590 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:29:13.713012 5051 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:13.713994 5906 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:13.844041 5906 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:13.844746 5906 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:13.847975 5906 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:13.850046 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:29:13.878460 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:29:13.883651 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:13.889526 5804 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:13.997611 6011 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:29:13.998116 6011 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:14.017336 6011 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:29:14.046607 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:29:14.048588 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:29:14.049672 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:14.050436 6011 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:29:14.060541 5804 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:14.061081 5804 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:14.063259 5804 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:14.068545 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:29:14.068650 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 6: this message will repeat every 5th retry.
W20260927 23:29:14.069228 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:29:14.069577 5051 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:14.075659 5527 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:14.083575 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:14.302827 5906 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:14.303357 5906 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:14.305495 5906 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:14.312014 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:29:14.314643 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:29:14.315026 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:14.315722 5527 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:14.366716 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 6: this message will repeat every 5th retry.
W20260927 23:29:14.486174 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:14.528509 5804 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:14.529013 5804 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:14.530982 5804 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:14.533370 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:29:14.533826 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:29:14.534165 5051 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:14.535491 5906 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:14.573215 6019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:29:14.573773 6019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:14.583356 6019 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:29:14.586612 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 6: this message will repeat every 5th retry.
W20260927 23:29:14.614167 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:29:14.614784 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:29:14.615165 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:14.615863 6019 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:29:14.718299 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 6: this message will repeat every 5th retry.
W20260927 23:29:15.053484 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:15.341567 5804 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:15.342020 5804 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:29:15.343541 5804 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:29:15.345247 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:29:15.345667 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:29:15.345965 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:15.346496 5804 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:15.347012 5527 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:15.347482 5527 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:29:15.349526 5527 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:29:15.351059 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:29:15.352643 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:29:15.354413 5054 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:15.355183 5527 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:15.669848 6025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:29:15.670284 6025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:15.671993 6025 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:29:15.677753 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:29:15.678730 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:29:15.679066 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:15.679677 6025 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:29:15.946211 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 6: this message will repeat every 5th retry.
W20260927 23:29:16.004184 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 6: this message will repeat every 5th retry.
I20260927 23:29:16.216885 5906 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:16.217417 5906 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:16.219247 5906 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:16.221011 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:29:16.221415 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:16.221767 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:16.222435 5906 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:16.467006 5906 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:16.467509 5906 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:16.470082 5906 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:16.471822 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:29:16.473654 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:16.474130 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:16.474939 5906 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:29:16.490733 5906 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:16.491211 5906 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:16.493024 5906 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:16.494540 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:29:16.494971 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:29:16.495258 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:16.495810 5906 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:16.652233 6035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:29:16.652882 6035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:16.655215 6035 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:29:16.662006 4615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111) [suppressed 9 similar messages]
W20260927 23:29:16.671532 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:29:16.673085 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:29:16.673427 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:16.674026 6035 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:29:16.873704 5906 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:16.874141 5906 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:16.875928 5906 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:16.877477 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:29:16.879451 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:29:16.879866 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:16.880856 5906 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:16.986034 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 327adcc2a27f42acb3a77d49fbf73796 consensus state has no leader
W20260927 23:29:17.110010 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 11: this message will repeat every 5th retry.
I20260927 23:29:17.112903 6029 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:17.113526 6029 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:17.116041 6029 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:17.118559 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:29:17.119068 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:29:17.119433 5051 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:17.120064 6029 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:17.219951 6029 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:17.220530 6029 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:17.222436 6029 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:17.224349 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:29:17.224812 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:29:17.225119 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:17.225659 6029 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:17.375080 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 11: this message will repeat every 5th retry.
I20260927 23:29:17.739907 5906 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:17.740365 5906 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:17.741878 5906 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:17.743191 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:29:17.743674 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:29:17.743963 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:17.744573 5906 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:17.788462 6044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:29:17.788897 6044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:17.790537 6044 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
I20260927 23:29:17.799232 6029 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:29:17.799746 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:29:17.799721 6029 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:17.801872 6029 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:17.803030 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:29:17.803437 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:17.804682 6044 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:29:17.805265 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:29:17.806267 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:29:17.806641 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:17.807338 6029 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:17.960276 6029 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:17.960717 6029 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:17.962265 6029 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:17.963673 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:29:17.964047 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:29:17.964370 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:17.964902 6029 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:18.079205 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 11: this message will repeat every 5th retry.
I20260927 23:29:18.120184 6051 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:18.120791 6051 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:18.123328 6051 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:18.124989 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:29:18.125435 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:29:18.125967 5051 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:18.127087 6029 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:18.646309 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 11: this message will repeat every 5th retry.
I20260927 23:29:18.648368 5906 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:18.648749 5906 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:18.650424 5906 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:18.651898 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:29:18.652612 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:29:18.652941 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:18.653458 5906 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:18.704803 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 1766d227273e4c7592c4f0a20f2f7b17 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1766d227273e4c7592c4f0a20f2f7b17 consensus state has no leader
I20260927 23:29:18.790637 6051 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:18.791059 6051 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:18.792770 6051 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:18.794064 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:29:18.794520 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:29:18.794821 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:18.795354 6051 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:19.152067 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 11: this message will repeat every 5th retry.
W20260927 23:29:19.202936 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 11: this message will repeat every 5th retry.
W20260927 23:29:19.227187 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 11: this message will repeat every 5th retry.
W20260927 23:29:19.687292 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 11: this message will repeat every 5th retry.
I20260927 23:29:19.854360 6051 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:19.854895 6051 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:19.856926 6051 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:19.858422 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:29:19.859160 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:29:19.859508 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:19.860210 6051 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:20.641693 6029 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:20.642354 6029 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:20.644376 6029 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:20.646271 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:29:20.646821 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:29:20.647074 5051 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:20.647684 6029 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:20.865471 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 11: this message will repeat every 5th retry.
W20260927 23:29:21.169432 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 11: this message will repeat every 5th retry.
W20260927 23:29:21.392609 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 11: this message will repeat every 5th retry.
I20260927 23:29:21.637207 6067 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:21.637684 6067 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:21.639510 6067 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:21.641294 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:29:21.641692 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:21.641966 5051 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:21.642490 6067 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:21.687768 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9f07e8a37404636bda765146b20fc60 consensus state has no leader
W20260927 23:29:21.945147 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 11: this message will repeat every 5th retry.
I20260927 23:29:22.515229 6058 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:22.515724 6058 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:22.518301 6058 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:22.519097 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:29:22.519749 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:22.520098 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:22.520730 6058 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:29:22.561067 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 11: this message will repeat every 5th retry.
W20260927 23:29:22.869891 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 11: this message will repeat every 5th retry.
I20260927 23:29:23.344631 6058 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:23.345156 6058 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:23.346956 6058 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:23.348562 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:29:23.348973 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:23.349288 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:23.349881 6058 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:23.351361 6086 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:29:23.351958 6086 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:23.354086 6086 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:29:23.356818 4615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111) [suppressed 4 similar messages]
W20260927 23:29:23.359994 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:29:23.360556 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:29:23.360906 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:23.361497 6086 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:29:23.405017 6058 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:23.405586 6058 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:23.410241 6058 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:23.417016 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:29:23.417879 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:29:23.418264 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:23.419276 6058 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:23.458192 6058 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:23.458698 6058 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:23.460629 6058 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:23.460837 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 11: this message will repeat every 5th retry.
W20260927 23:29:23.462617 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:29:23.463034 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:29:23.463380 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:23.464335 6058 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:23.502697 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 11: this message will repeat every 5th retry.
I20260927 23:29:23.743331 6067 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:23.743817 6067 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:23.745580 6067 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:23.747047 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:29:23.747501 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:29:23.747792 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:23.748394 6067 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:23.912510 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 11: this message will repeat every 5th retry.
I20260927 23:29:23.932225 6067 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:23.932936 6067 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:23.935318 6067 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:23.936686 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:29:23.937112 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:29:23.937394 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:23.937906 6067 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:24.463075 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 11: this message will repeat every 5th retry.
W20260927 23:29:24.609618 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 11: this message will repeat every 5th retry.
I20260927 23:29:24.843456 6067 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:24.844033 6067 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:29:24.846222 6067 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:29:24.847677 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:29:24.848438 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:29:24.848783 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:24.850001 6067 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:24.850425 6058 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:24.850811 6058 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:24.852530 6058 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:24.853884 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:29:24.854326 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:29:24.854607 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:24.855115 6058 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:24.990064 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 11: this message will repeat every 5th retry.
W20260927 23:29:25.038640 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 11: this message will repeat every 5th retry.
W20260927 23:29:25.188766 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 11: this message will repeat every 5th retry.
W20260927 23:29:26.182572 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 11: this message will repeat every 5th retry.
W20260927 23:29:26.187435 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 11: this message will repeat every 5th retry.
I20260927 23:29:26.832054 6067 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:26.832616 6067 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:26.834650 6067 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:26.836447 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:29:26.839192 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:29:26.839619 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:26.840233 6067 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:26.943020 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:27.112197 6083 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:27.112895 6083 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:27.115798 6083 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:27.116825 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:29:27.120218 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:29:27.120641 5051 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:27.121309 6112 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:27.371825 4615 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ac88a4e4bee64586bef7b6b409a63f4c consensus state has no leader
W20260927 23:29:27.494737 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:27.511164 6067 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:27.511727 6067 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:27.513861 6067 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:27.516038 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:29:27.518515 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:29:27.518905 5051 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:27.519572 6067 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:27.924149 6112 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:27.924691 6112 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:29:27.928206 6112 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:29:27.928234 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:29:27.929232 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:29:27.929665 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:27.930488 6083 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:27.934832 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 16: this message will repeat every 5th retry.
W20260927 23:29:28.225342 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 16: this message will repeat every 5th retry.
W20260927 23:29:28.881184 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:29.149914 6112 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:29.150470 6112 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:29.152539 6112 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:29.155452 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:29:29.156533 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:29.156862 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:29.157467 6112 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:29:29.398976 6067 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:29.399549 6067 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:29.401518 6067 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:29.403144 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:29:29.403586 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:29:29.403913 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:29.404525 6067 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:29.454001 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 16: this message will repeat every 5th retry.
W20260927 23:29:29.491153 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 16: this message will repeat every 5th retry.
W20260927 23:29:29.739467 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:30.041100 6128 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:30.041620 6128 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:30.043185 6128 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:30.044813 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:29:30.045199 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:29:30.045473 5051 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:30.046000 6128 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:30.391858 6121 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:30.392343 6121 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:30.393006 6127 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:30.393424 6127 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:30.394506 6121 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:29:30.395311 6127 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:30.396353 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:29:30.397238 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:29:30.397330 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:29:30.397836 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:30.398476 6127 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:30.398718 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:29:30.399075 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:30.399580 6127 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:30.555523 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:30.678049 6127 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:30.678750 6127 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:30.680938 6127 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:30.682901 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:29:30.683421 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:29:30.683720 5051 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:30.684540 6127 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:30.687242 5015 debug-util.cc:398] Leaking SignalData structure 0x7b08000e8a20 after lost signal to thread 4889
W20260927 23:29:30.688453 5015 debug-util.cc:398] Leaking SignalData structure 0x7b080012e1a0 after lost signal to thread 5018
I20260927 23:29:30.784642 6127 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:30.785261 6127 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:30.787464 6127 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:30.789791 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:29:30.791427 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:29:30.791955 5051 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:30.792886 6128 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:30.875867 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 16: this message will repeat every 5th retry.
W20260927 23:29:31.440900 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:31.513190 6139 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:29:31.513700 6139 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:31.515797 6139 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:29:31.527594 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:29:31.533299 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:29:31.533917 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:29:31.534273 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:31.534984 6139 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:29:32.305585 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:32.310161 6128 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:32.310704 6128 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:32.313653 6128 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:32.314283 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:29:32.316447 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:29:32.316769 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:32.317402 6128 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:32.627058 6128 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:32.627563 6128 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:32.632190 6128 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:32.632548 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:29:32.634364 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:29:32.634815 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:32.654520 6128 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:32.728478 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 16: this message will repeat every 5th retry.
W20260927 23:29:32.914805 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 16: this message will repeat every 5th retry.
W20260927 23:29:33.262015 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 16: this message will repeat every 5th retry.
W20260927 23:29:33.384949 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 327adcc2a27f42acb3a77d49fbf73796 consensus state has no leader
W20260927 23:29:33.463794 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:33.516449 6153 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:33.516935 6153 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:33.519403 6153 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:33.521973 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:29:33.523773 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:29:33.524174 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:33.525723 6153 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:33.662405 6153 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:33.662940 6153 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:33.664615 6153 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:33.666164 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:29:33.666642 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:29:33.666922 5051 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:33.667477 6153 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:34.505421 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:34.600111 6153 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:34.600715 6153 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:34.603955 6153 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:34.605208 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:29:34.606635 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:34.607028 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:34.607623 6153 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:34.652426 6152 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:34.652876 6152 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:34.654373 6152 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:34.655922 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:29:34.656353 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:29:34.656618 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:34.657160 6152 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:34.703521 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 16: this message will repeat every 5th retry.
W20260927 23:29:34.751226 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:34.869721 6164 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:34.872641 6164 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:34.874830 6164 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:34.876374 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:29:34.876801 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:29:34.877174 5051 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:34.877875 6164 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:34.895850 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 16: this message will repeat every 5th retry.
W20260927 23:29:35.089815 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 1766d227273e4c7592c4f0a20f2f7b17 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1766d227273e4c7592c4f0a20f2f7b17 consensus state has no leader
W20260927 23:29:35.183630 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 16: this message will repeat every 5th retry.
W20260927 23:29:35.193951 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:35.350703 6164 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:35.351298 6164 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:35.353443 6164 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:35.355116 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:29:35.355548 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:29:35.355834 5051 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:35.356406 6164 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:35.483218 6152 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:35.483862 6152 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:35.485590 6152 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:35.487139 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:29:35.487547 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:29:35.487838 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:35.488420 6152 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:35.638325 6153 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:35.638922 6153 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:35.641166 6153 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:35.643049 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:29:35.643494 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:29:35.643831 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:35.645053 6112 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:35.851074 6153 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:35.851553 6153 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:29:35.853847 6153 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:29:35.854784 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:29:35.855239 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:29:35.855593 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:35.856153 6153 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:35.883769 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:36.091629 6112 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:36.092150 6112 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:36.093955 6112 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:36.095574 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:29:36.095986 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:29:36.096376 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:36.096939 6112 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:36.168025 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 16: this message will repeat every 5th retry.
I20260927 23:29:36.581205 6153 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:36.581682 6153 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:29:36.583364 6153 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:29:36.584872 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:29:36.585273 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:29:36.585623 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:36.586148 6153 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:36.944245 6153 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:36.944784 6153 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:36.946765 6153 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:36.948699 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:29:36.949724 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:36.950103 5051 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:36.950752 6153 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:36.974284 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:37.833616 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:37.905969 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:38.081149 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9f07e8a37404636bda765146b20fc60 consensus state has no leader
I20260927 23:29:38.142707 6186 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:38.143216 6186 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:38.145037 6186 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:38.146458 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:29:38.146823 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:29:38.147199 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:38.147853 6186 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:38.251603 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:38.740546 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:38.757714 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 21: this message will repeat every 5th retry.
I20260927 23:29:38.835132 6186 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:38.835664 6186 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:38.837427 6186 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:38.838985 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:29:38.839421 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:29:38.839710 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:38.840257 6186 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:38.954408 6186 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:38.955010 6186 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:38.957728 6186 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:38.959308 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:29:38.959709 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:38.960037 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:38.960763 6186 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:29:39.480015 6186 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:39.480522 6186 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:39.482244 6186 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:39.483899 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:29:39.484449 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:29:39.484812 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:39.485870 6186 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:39.645219 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:39.893160 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 21: this message will repeat every 5th retry.
I20260927 23:29:40.081844 6206 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:40.082574 6206 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:40.084738 6206 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:40.085813 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:29:40.086719 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:29:40.087085 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:40.087767 6206 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:40.115769 6206 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:40.116333 6206 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:40.118002 6206 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:40.119859 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:29:40.120342 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:29:40.120707 5051 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:40.121358 6206 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:40.399041 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:40.653786 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:40.676055 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 21: this message will repeat every 5th retry.
I20260927 23:29:40.900588 6212 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:40.901398 6212 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:40.904088 6212 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:40.906116 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:29:40.908233 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:40.909425 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:40.910213 6212 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:42.044703 6212 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:42.045222 6212 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:42.047039 6212 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:42.048928 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:29:42.049336 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:29:42.049715 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:42.050550 6212 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:42.153519 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:42.515555 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:42.746723 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:42.831094 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 21: this message will repeat every 5th retry.
I20260927 23:29:43.134848 6242 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:29:43.135717 6242 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:43.138939 6242 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:29:43.141237 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:29:43.149341 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:29:43.150486 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:29:43.150897 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:43.151650 6242 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:29:43.336254 4874 debug-util.cc:398] Leaking SignalData structure 0x7b08000b4e60 after lost signal to thread 4744
W20260927 23:29:43.337798 4874 debug-util.cc:398] Leaking SignalData structure 0x7b0800122680 after lost signal to thread 4877
W20260927 23:29:43.839341 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:43.975023 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 21: this message will repeat every 5th retry.
I20260927 23:29:44.488808 6164 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:44.489336 6164 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:44.491261 6164 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:44.492560 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:29:44.494410 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:29:44.494728 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:44.495297 6164 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:44.522132 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:44.608786 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:45.066696 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 21: this message will repeat every 5th retry.
I20260927 23:29:45.127063 6164 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:45.127573 6164 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:45.129585 6164 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:45.132637 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:29:45.134658 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:29:45.135084 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:45.136545 6164 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:45.290061 6265 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:45.290630 6265 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:45.292745 6265 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:45.294605 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:45.296370 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:29:45.298364 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:29:45.298765 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:45.299496 6241 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:45.321401 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 21: this message will repeat every 5th retry.
I20260927 23:29:45.910518 6250 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:45.911067 6250 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:45.913127 6250 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:45.914949 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:29:45.916448 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:29:45.916822 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:45.917593 6250 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:46.205982 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 21: this message will repeat every 5th retry.
I20260927 23:29:46.216233 6250 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:46.216846 6250 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:46.218432 6250 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:46.219751 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:29:46.220206 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:29:46.220515 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:46.221058 6250 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:46.267235 6250 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:46.267740 6250 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:46.269570 6250 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:46.271114 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:29:46.271544 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:29:46.271883 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:46.272560 6250 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:46.300942 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 21: this message will repeat every 5th retry.
W20260927 23:29:46.474864 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 26: this message will repeat every 5th retry.
I20260927 23:29:47.509087 6250 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:47.509604 6250 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:47.511323 6250 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:47.512861 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:29:47.513334 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:47.513619 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:47.514346 6250 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:47.701129 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 26: this message will repeat every 5th retry.
I20260927 23:29:47.751593 6264 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:47.752002 6264 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:47.753597 6264 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:47.754958 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:29:47.755349 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:47.755661 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:47.756184 6264 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:48.107049 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:48.189249 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:48.649632 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 26: this message will repeat every 5th retry.
I20260927 23:29:48.901595 6293 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:48.902174 6293 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:48.904261 6293 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:48.905944 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:29:48.906904 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:29:48.907277 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:48.908082 6293 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:29:49.108548 6301 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:49.109079 6301 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:49.110742 6301 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:49.112169 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:29:49.112571 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:29:49.112859 5051 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:49.113510 6301 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:49.162918 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:49.363252 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 26: this message will repeat every 5th retry.
I20260927 23:29:49.679183 6294 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:49.679749 6294 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:49.681560 6294 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:49.683370 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:29:49.683756 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:29:49.684053 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:49.684729 6294 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:49.826874 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:50.030287 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:50.135833 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:50.325165 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 26: this message will repeat every 5th retry.
I20260927 23:29:50.870011 6294 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:50.870749 6294 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:29:50.875111 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:50.875972 6294 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:50.877031 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:29:50.877650 5051 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:50.878744 6294 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:51.258559 6294 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:51.259109 6294 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:51.261299 6294 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:51.263059 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:29:51.263778 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:29:51.264213 5051 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:51.265424 6294 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:51.665920 5165 heartbeater.cc:644] Failed to heartbeat to 127.2.57.62:35155 (924 consecutive failures): Network error: Failed to ping master at 127.2.57.62:35155: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:29:51.757500 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:51.794036 5021 heartbeater.cc:644] Failed to heartbeat to 127.2.57.62:35155 (77 consecutive failures): Network error: Failed to ping master at 127.2.57.62:35155: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:29:52.170941 4879 heartbeater.cc:644] Failed to heartbeat to 127.2.57.61:46527 (84 consecutive failures): Network error: Failed to ping master at 127.2.57.61:46527: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:29:52.225467 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 26: this message will repeat every 5th retry.
I20260927 23:29:52.256078 6294 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:52.256572 6294 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:29:52.258333 6294 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:29:52.259759 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:29:52.260147 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:29:52.260488 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:52.261116 6294 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:52.569694 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:52.715603 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 26: this message will repeat every 5th retry.
I20260927 23:29:52.763536 6294 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:52.764079 6294 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:52.765872 6294 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:52.767369 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:29:52.768154 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:29:52.768564 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:52.769255 6294 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:53.261143 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 26: this message will repeat every 5th retry.
I20260927 23:29:53.319824 6316 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:53.320485 6316 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:53.322235 6316 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:53.324105 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:29:53.324592 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:29:53.324947 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:53.325881 6311 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:53.433885 6311 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:53.434316 6311 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:53.435971 6311 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:53.437289 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:29:53.437698 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:29:53.437978 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:53.438514 6311 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:53.644107 6311 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:53.644847 6311 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:53.646749 6311 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:53.648663 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:29:53.649631 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:29:53.650092 5051 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:53.651079 6311 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:53.767246 6316 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:53.767696 6316 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:53.769503 6316 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:53.773005 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:29:53.775913 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:29:53.776337 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:53.777320 6311 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:53.789538 6311 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:53.789959 6311 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:53.791455 6311 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:53.792906 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:29:53.793334 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:29:53.793601 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:53.794139 6311 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:53.925685 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 26: this message will repeat every 5th retry.
I20260927 23:29:54.240689 6325 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:54.241178 6325 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:54.243444 6325 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:54.245160 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:29:54.245638 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:29:54.245975 5051 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:54.246657 6325 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:54.605088 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:54.641248 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:54.692672 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 26: this message will repeat every 5th retry.
I20260927 23:29:55.028496 6343 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:55.029012 6343 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:29:55.031122 6343 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:29:55.033527 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:29:55.034083 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:29:55.034462 5054 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:55.035181 6343 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:55.188009 6343 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:55.188639 6343 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:29:55.191234 6343 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:55.194994 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:29:55.195528 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:29:55.195811 5051 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:55.196434 6343 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:55.211849 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:55.286362 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:55.898195 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 26: this message will repeat every 5th retry.
I20260927 23:29:56.253576 6316 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:56.254032 6316 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:56.255744 6316 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:56.257182 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:29:56.257607 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:29:56.257879 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:56.258426 6316 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:56.847990 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 26: this message will repeat every 5th retry.
W20260927 23:29:57.062667 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 31: this message will repeat every 5th retry.
I20260927 23:29:57.406050 6349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:29:57.406809 6349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:29:57.408934 6349 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:29:57.411937 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:29:57.416159 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:29:57.416600 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:29:57.416903 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:29:57.417495 6349 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:29:57.790218 6325 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:57.790782 6325 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:57.792584 6325 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:57.794253 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:29:57.794742 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:29:57.795125 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:57.796209 6343 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:29:58.030251 6356 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:58.030841 6356 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:58.035452 6356 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:58.043815 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:29:58.081218 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:29:58.081607 5051 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:58.082377 6357 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:58.135830 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 31: this message will repeat every 5th retry.
I20260927 23:29:58.169356 6356 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:29:58.169848 6356 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:29:58.171525 6356 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:29:58.172868 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:29:58.173291 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:29:58.173544 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:29:58.174027 6357 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:29:58.269793 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:29:58.345090 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:29:58.503355 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:29:58.992564 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:29:59.012861 5159 debug-util.cc:398] Leaking SignalData structure 0x7b080013f3a0 after lost signal to thread 5032
W20260927 23:29:59.014369 5159 debug-util.cc:398] Leaking SignalData structure 0x7b08000d8320 after lost signal to thread 5162
W20260927 23:29:59.243592 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:29:59.257771 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:29:59.366102 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:29:59.690518 4661 debug-util.cc:398] Leaking SignalData structure 0x7b08000b74a0 after lost signal to thread 4593
W20260927 23:29:59.691346 4661 debug-util.cc:398] Leaking SignalData structure 0x7b08000bc800 after lost signal to thread 4664
W20260927 23:30:00.020071 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:30:00.169368 4615 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ac88a4e4bee64586bef7b6b409a63f4c consensus state has no leader
W20260927 23:30:00.682945 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:30:01.349321 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 31: this message will repeat every 5th retry.
I20260927 23:30:01.700538 6374 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:01.701176 6374 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:01.703521 6374 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:01.705111 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:30:01.705724 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:30:01.706050 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:01.706635 6374 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:01.758462 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 31: this message will repeat every 5th retry.
I20260927 23:30:01.911203 6365 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:01.911736 6365 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:01.913980 6365 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:01.915239 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:30:01.915961 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:30:01.916368 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:01.917076 6365 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:01.921974 6365 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:01.922515 6365 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:01.924672 6365 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:01.926329 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:30:01.926828 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:30:01.927197 5051 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:01.927925 6365 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:02.514703 6374 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:02.515262 6374 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:02.517076 6374 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:02.518605 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:30:02.519081 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:30:02.519471 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:02.520247 6374 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:30:02.545766 6374 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:02.546427 6374 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:02.549350 6374 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:02.550176 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:30:02.550719 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:30:02.551095 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:02.551784 6374 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:02.670441 6377 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:02.671015 6377 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:02.672881 6377 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:02.674575 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:30:02.675014 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:30:02.675402 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:02.676133 6377 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:02.882977 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 31: this message will repeat every 5th retry.
I20260927 23:30:02.908699 6374 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:02.909185 6374 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:02.911044 6374 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:02.912955 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:30:02.913409 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:30:02.913733 5051 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:02.914826 6374 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:03.349210 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 31: this message will repeat every 5th retry.
I20260927 23:30:03.571729 6365 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:03.572283 6365 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:03.574373 6365 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:03.575779 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:30:03.576277 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:30:03.576725 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:03.577456 6365 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:03.735607 6374 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:03.736038 6374 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:03.737784 6374 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:03.739296 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:30:03.739743 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:30:03.740146 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:03.740938 6377 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:03.770915 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:30:04.284260 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:30:04.639389 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:30:04.803894 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 31: this message will repeat every 5th retry.
I20260927 23:30:04.844602 6365 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:04.845024 6365 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:04.846465 6365 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:04.847986 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:30:04.848441 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:30:04.848717 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:04.849195 6365 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:04.884248 6365 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:04.884750 6365 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:30:04.886456 6365 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:30:04.888451 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:30:04.888952 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:30:04.889258 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:04.889933 6365 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:04.899595 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 31: this message will repeat every 5th retry.
I20260927 23:30:04.910357 6365 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:04.910876 6365 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:04.912632 6365 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:04.913973 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:30:04.914350 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:30:04.914606 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:04.915112 6365 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:05.099668 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 31: this message will repeat every 5th retry.
I20260927 23:30:05.369350 6365 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:05.369910 6365 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:05.371927 6365 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:05.373299 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:30:05.373757 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:30:05.374023 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:05.374534 6365 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:05.601604 5015 debug-util.cc:398] Leaking SignalData structure 0x7b0800131f80 after lost signal to thread 4889
W20260927 23:30:05.602737 5015 debug-util.cc:398] Leaking SignalData structure 0x7b080010e000 after lost signal to thread 5018
W20260927 23:30:05.623312 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:30:05.680718 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:30:06.188019 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 327adcc2a27f42acb3a77d49fbf73796 consensus state has no leader
I20260927 23:30:06.318328 6374 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:06.318818 6374 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:06.320680 6374 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:06.322556 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:30:06.323091 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:30:06.323474 5051 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:06.324028 6374 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:06.696102 6365 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:06.696755 6365 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:06.698428 6365 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:06.699971 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:30:06.701097 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:30:06.701440 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:06.702062 6365 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:06.947337 6386 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:06.947925 6386 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:06.950151 6386 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:06.952127 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:30:06.953189 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:30:06.953554 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:06.955601 6386 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:06.961728 6386 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:06.962230 6386 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:06.964284 6386 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:06.965813 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:30:06.966279 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:30:06.966595 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:06.967190 6386 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:07.150856 6386 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:07.151335 6386 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:07.154270 6386 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:07.155004 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:30:07.155419 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:30:07.155695 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:07.156234 6386 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:07.162941 6386 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:07.163426 6386 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:07.165246 6386 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:07.166640 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:30:07.167063 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:30:07.167336 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:07.167861 6386 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:07.319880 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 31: this message will repeat every 5th retry.
W20260927 23:30:07.524019 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 36: this message will repeat every 5th retry.
W20260927 23:30:07.739048 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 36: this message will repeat every 5th retry.
W20260927 23:30:07.810484 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 36: this message will repeat every 5th retry.
W20260927 23:30:07.862047 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 1766d227273e4c7592c4f0a20f2f7b17 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1766d227273e4c7592c4f0a20f2f7b17 consensus state has no leader
I20260927 23:30:07.969583 6386 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:07.970041 6386 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:07.971719 6386 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:07.973433 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:30:07.973877 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:30:07.974185 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:07.974736 6386 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:08.440989 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 36: this message will repeat every 5th retry.
W20260927 23:30:08.520661 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 36: this message will repeat every 5th retry.
W20260927 23:30:08.526357 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:08.740269 6386 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:08.740813 6386 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:30:08.742753 6386 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:30:08.745242 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:30:08.746809 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:30:08.748633 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:08.749413 6386 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:09.215407 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 36: this message will repeat every 5th retry.
W20260927 23:30:09.247982 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:09.471534 6422 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:09.472050 6422 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:09.474821 6422 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:09.481835 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:30:09.488683 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:30:09.489315 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:09.490343 6422 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:09.615417 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:09.661847 6422 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:09.662362 6422 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:09.664494 6422 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:09.665678 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:30:09.666461 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:30:09.666721 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:09.667290 6422 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:09.896914 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 36: this message will repeat every 5th retry.
W20260927 23:30:10.079859 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:10.544858 6422 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:10.545434 6422 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:10.547523 6422 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:10.549307 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:30:10.549816 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:30:10.550207 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:10.550938 6422 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:10.585067 6422 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:10.585639 6422 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:10.587373 6422 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:10.588981 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:30:10.589389 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:30:10.589670 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:10.590286 6422 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:10.854638 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9f07e8a37404636bda765146b20fc60 consensus state has no leader
W20260927 23:30:10.898509 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:10.994562 6420 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:10.995219 6420 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:30:10.997677 6420 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:30:10.999243 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:30:10.999800 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:30:11.000089 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:11.000816 6420 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:11.069325 6430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:30:11.069902 6430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:30:11.071674 6430 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:30:11.074450 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:30:11.077414 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:30:11.077944 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:30:11.078353 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:30:11.079010 6430 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:30:11.826383 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:12.424722 6420 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:12.425192 6420 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:12.426849 6420 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:12.428418 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:30:12.429956 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:30:12.430353 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:12.431026 6420 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:12.523615 6445 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:12.524374 6445 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:12.526340 6445 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:12.527596 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:30:12.528033 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:30:12.528422 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:12.529026 6445 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:12.774950 6445 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:12.775463 6445 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:30:12.777488 6445 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:30:12.778834 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:30:12.780260 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:30:12.780920 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:12.781523 6445 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:13.387658 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 36: this message will repeat every 5th retry.
W20260927 23:30:13.578051 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:13.621115 6451 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:13.621707 6451 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:13.623699 6451 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:13.625243 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:30:13.625656 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:30:13.625970 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:13.626662 6451 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:13.706175 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:14.049882 6445 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:14.050457 6445 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:14.052017 6445 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:14.053617 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:30:14.054064 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:30:14.054365 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:14.054862 6445 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:14.080614 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:14.121308 6451 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:14.121778 6451 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:30:14.123347 6451 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:30:14.124760 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:30:14.125195 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:30:14.125613 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:14.126418 6445 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:14.330276 6445 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:14.330706 6445 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:14.332427 6445 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:14.333822 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:30:14.334393 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:30:14.334681 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:14.335276 6445 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:14.488814 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 36: this message will repeat every 5th retry.
W20260927 23:30:14.508620 5015 debug-util.cc:398] Leaking SignalData structure 0x7b08000ff240 after lost signal to thread 4889
W20260927 23:30:14.510047 5015 debug-util.cc:398] Leaking SignalData structure 0x7b08001273c0 after lost signal to thread 5018
I20260927 23:30:14.759831 6463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:30:14.760648 6463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:30:14.763531 6463 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:30:14.772503 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:30:14.773057 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:30:14.773442 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:30:14.774026 6463 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:30:14.871250 6462 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:14.871734 6462 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:14.874032 6462 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:14.876670 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:30:14.877614 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:30:14.878013 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:14.878561 6462 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:14.885931 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 36: this message will repeat every 5th retry.
W20260927 23:30:15.148576 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:15.180411 6451 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:15.180933 6451 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:15.182917 6451 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:15.184588 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:30:15.185201 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:30:15.185500 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:15.186080 6451 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:15.293637 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:15.443019 6451 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:15.443536 6451 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:15.445494 6451 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:15.448877 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:30:15.451419 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:30:15.451838 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:15.452600 6445 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:15.522212 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:16.017983 6468 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:16.018432 6468 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:16.020035 6468 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:16.021884 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:30:16.022296 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:30:16.022621 5051 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:16.023260 6468 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:16.080639 6462 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:16.081135 6462 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:30:16.083037 6462 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:30:16.086287 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:30:16.088429 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:30:16.088757 5054 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:16.089434 6468 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:16.295264 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:16.906111 6462 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:16.906612 6462 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:16.908418 6462 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:16.909942 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:30:16.910341 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:30:16.910619 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:16.911164 6462 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:17.059770 6468 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:17.060376 6468 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:17.062340 6468 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:17.063964 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:30:17.064366 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:30:17.064769 5051 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:17.065526 6468 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:17.335779 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:17.360167 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:17.376277 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:17.760244 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 41: this message will repeat every 5th retry.
I20260927 23:30:17.805485 6468 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:17.805968 6468 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:17.807597 6468 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:17.809226 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:30:17.809723 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:30:17.810051 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:17.810698 6468 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:17.816334 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 36: this message will repeat every 5th retry.
I20260927 23:30:18.015480 6468 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:18.015973 6468 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:18.017692 6468 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:18.019264 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:30:18.019662 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:30:18.020020 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:18.020596 6468 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:18.185973 6462 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:18.186540 6462 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:18.188655 6462 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:18.190433 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:30:18.190958 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:30:18.191322 5051 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:18.192060 6462 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:18.581707 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:18.646495 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:18.807605 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 41: this message will repeat every 5th retry.
I20260927 23:30:19.054777 6468 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:19.055243 6468 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:19.056875 6468 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:19.058264 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:30:19.058620 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:30:19.058900 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:19.059515 6468 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:19.097741 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:19.257685 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:19.375967 4661 debug-util.cc:398] Leaking SignalData structure 0x7b08000b2c00 after lost signal to thread 4593
W20260927 23:30:19.376637 4661 debug-util.cc:398] Leaking SignalData structure 0x7b08000b38a0 after lost signal to thread 4664
I20260927 23:30:19.817842 6445 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:19.818336 6445 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:19.820137 6445 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:19.821461 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:30:19.822013 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:30:19.822325 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:19.822896 6445 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:19.866945 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 41: this message will repeat every 5th retry.
I20260927 23:30:19.892670 6445 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:19.893252 6445 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:19.895535 6445 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:19.897491 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:30:19.898849 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:30:19.899158 5051 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:19.899780 6445 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:20.361158 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 41: this message will repeat every 5th retry.
I20260927 23:30:20.567386 6500 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:20.567900 6500 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:20.570199 6500 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:20.571942 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:30:20.572822 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:30:20.573141 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:20.573711 6500 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:20.607898 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 41: this message will repeat every 5th retry.
I20260927 23:30:20.608098 6468 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:20.608664 6468 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:20.610714 6468 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:20.613773 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:30:20.614347 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:30:20.614696 5051 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:20.615412 6468 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:30:20.794526 6495 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:20.795799 6495 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:20.800056 6495 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:20.802197 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:30:20.802977 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:30:20.803319 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:20.804032 6495 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:21.528817 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 41: this message will repeat every 5th retry.
I20260927 23:30:21.875109 6505 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:21.875588 6505 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:21.877359 6505 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:21.879004 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:30:21.879571 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:30:21.879951 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:21.880637 6505 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:23.445211 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 41: this message will repeat every 5th retry.
I20260927 23:30:23.771152 6468 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:23.771580 6468 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:23.773449 6468 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:23.775610 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:30:23.776500 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:30:23.776949 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:23.777892 6516 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:23.786677 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:23.836484 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:23.984879 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:24.212173 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 41: this message will repeat every 5th retry.
I20260927 23:30:24.351972 6517 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:24.352483 6517 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:24.354388 6517 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:24.355942 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:30:24.357059 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:30:24.357316 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:24.358331 6517 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:24.533329 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:24.816934 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 41: this message will repeat every 5th retry.
I20260927 23:30:25.173369 6517 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:25.173880 6517 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:25.177289 6517 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:25.178433 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:30:25.178858 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:30:25.179206 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:25.179759 6517 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:25.458531 6523 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:25.459090 6523 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:25.461465 6523 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:25.462769 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:30:25.463457 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:30:25.463744 5051 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:25.464385 6523 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:25.629190 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:25.693640 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:26.056813 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 41: this message will repeat every 5th retry.
I20260927 23:30:26.227999 6530 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:26.228691 6530 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:26.231640 6530 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:26.232290 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:30:26.233476 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:30:26.233851 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:26.234470 6530 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:26.587664 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 46: this message will repeat every 5th retry.
I20260927 23:30:26.716657 6530 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:26.717186 6530 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:26.719122 6530 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:26.721102 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:30:26.721684 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:30:26.722062 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:26.722759 6530 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:27.363119 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:27.698168 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:27.706445 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:27.934332 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 46: this message will repeat every 5th retry.
I20260927 23:30:28.240406 6546 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:28.240885 6546 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:28.242607 6546 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:28.244514 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:30:28.244999 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:30:28.245297 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:28.245909 6546 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:28.258172 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 41: this message will repeat every 5th retry.
W20260927 23:30:28.328333 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:28.395394 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:28.987165 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:29.007519 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:29.580406 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 46: this message will repeat every 5th retry.
I20260927 23:30:30.435643 6546 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:30.436170 6546 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:30:30.438264 6546 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:30:30.439854 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:30:30.440251 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:30:30.440641 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:30.441300 6546 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:30.574301 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:31.088673 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 46: this message will repeat every 5th retry.
I20260927 23:30:31.250860 6557 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:31.251737 6557 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:31.254271 6557 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:31.255884 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:30:31.256594 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:30:31.256994 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:31.257794 6557 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:31.524637 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 46: this message will repeat every 5th retry.
I20260927 23:30:31.911515 6562 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:31.911999 6562 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:31.914098 6562 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:31.915611 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:30:31.916239 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:30:31.916591 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:31.917312 6562 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:32.222409 6562 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:32.223012 6562 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:32.224913 6562 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:32.226504 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:30:32.226969 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:30:32.227265 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:32.227891 6562 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:33.161338 6577 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:33.161859 6577 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:33.164115 6577 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:33.167290 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:30:33.168059 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:30:33.168481 5051 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:33.169535 6577 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:33.221508 6577 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:33.222097 6577 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:30:33.228634 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:30:33.234602 6577 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:33.236814 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:30:33.237361 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:33.238094 6577 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:33.274358 6577 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:33.274843 6577 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:33.276674 6577 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:33.278563 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:30:33.279229 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:30:33.279520 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:33.280207 6577 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:33.392266 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 46: this message will repeat every 5th retry.
I20260927 23:30:33.449997 6577 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:33.450444 6577 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:33.452253 6577 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:33.453509 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:30:33.453948 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:30:33.454239 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:33.454777 6577 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:33.506482 6580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:30:33.507061 6580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:30:33.509502 6580 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:30:33.513195 4615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260927 23:30:33.516286 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:30:33.517031 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:30:33.517336 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:30:33.518111 6580 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:30:33.601459 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 46: this message will repeat every 5th retry.
I20260927 23:30:33.724357 6557 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:33.724933 6557 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:33.727522 6557 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:33.731513 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:30:33.733106 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:30:33.733510 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:33.734714 6571 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:33.814383 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:33.893883 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:34.056437 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 46: this message will repeat every 5th retry.
I20260927 23:30:34.447705 6577 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:34.448216 6577 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:30:34.449975 6577 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:30:34.451619 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:30:34.452137 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:30:34.452562 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:34.453274 6577 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:34.607916 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:34.865064 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:35.341917 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:35.714617 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 46: this message will repeat every 5th retry.
I20260927 23:30:36.062151 6557 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:36.062630 6557 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:36.064451 6557 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:36.066282 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:30:36.066670 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:30:36.066948 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:36.067548 6557 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:36.155880 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:36.646261 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 51: this message will repeat every 5th retry.
I20260927 23:30:36.830211 6591 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:36.830670 6591 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:36.832544 6591 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:36.834100 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:30:36.834542 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:30:36.834894 5051 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:36.836084 6557 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:36.842099 6557 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:36.842505 6557 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:36.844067 6557 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:36.845517 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:30:36.845963 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:30:36.846289 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:36.846832 6557 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:30:37.293111 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 51: this message will repeat every 5th retry.
I20260927 23:30:37.682087 6591 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:37.682626 6591 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:37.684696 6591 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:37.686174 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:30:37.687561 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:30:37.688002 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:37.688696 6591 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:37.764973 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:37.889657 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:37.929143 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:37.963397 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:37.965898 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 51: this message will repeat every 5th retry.
I20260927 23:30:38.235558 6591 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:38.236115 6591 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:38.238274 6591 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:38.239888 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:30:38.240396 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:30:38.240780 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:38.241541 6591 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:38.680589 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 46: this message will repeat every 5th retry.
W20260927 23:30:38.710438 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:38.744905 4661 debug-util.cc:398] Leaking SignalData structure 0x7b08000c47e0 after lost signal to thread 4593
W20260927 23:30:38.745738 4661 debug-util.cc:398] Leaking SignalData structure 0x7b080004b3a0 after lost signal to thread 4664
I20260927 23:30:39.090656 6597 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:39.091135 6597 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:39.092862 6597 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:39.094448 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:30:39.094871 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:30:39.095129 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:39.095640 6597 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:39.539466 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 51: this message will repeat every 5th retry.
I20260927 23:30:39.692196 6591 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:39.692809 6591 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:39.694412 6591 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:39.695920 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:30:39.696372 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:30:39.696650 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:39.697180 6591 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:39.727051 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 51: this message will repeat every 5th retry.
I20260927 23:30:40.531802 6597 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:40.532500 6597 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:30:40.535220 6597 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:30:40.536763 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:30:40.537541 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:30:40.537910 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:40.538587 6597 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:40.722818 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:41.087007 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:41.269091 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 51: this message will repeat every 5th retry.
I20260927 23:30:42.173903 6597 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:42.174563 6597 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:42.176734 6597 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:42.178543 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:30:42.179157 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:30:42.179544 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:42.180341 6597 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:42.189296 6597 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:42.189822 6597 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:42.191794 6597 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:42.193167 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:30:42.193881 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:30:42.194180 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:42.194728 6597 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:42.498536 6597 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:42.498972 6597 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:42.500928 6597 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:42.502110 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:30:42.502588 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:30:42.502882 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:42.503445 6597 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:42.878829 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 51: this message will repeat every 5th retry.
I20260927 23:30:43.163106 6609 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:43.163614 6609 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:43.165570 6609 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:43.166985 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:30:43.167387 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:30:43.167734 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:43.168352 6609 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:43.594359 6598 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:43.594890 6598 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:43.597478 6598 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:43.598815 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:30:43.599272 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:30:43.599558 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:43.600135 6598 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:43.644111 4619 maintenance_manager.cc:419] P 83cd22a89b7846999c0e4dfab80ebe9d: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033368
I20260927 23:30:43.752280 4618 maintenance_manager.cc:643] P 83cd22a89b7846999c0e4dfab80ebe9d: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.107s user 0.072s sys 0.028s Metrics: {"bytes_written":32408,"cfile_init":1,"drs_written":1,"lbm_read_time_us":277,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3046,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":82,"thread_start_us":819,"threads_started":1,"wal-append.queue_time_us":1168}
I20260927 23:30:43.754547 4619 maintenance_manager.cc:419] P 83cd22a89b7846999c0e4dfab80ebe9d: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 4886 bytes on disk
I20260927 23:30:43.756243 4618 maintenance_manager.cc:643] P 83cd22a89b7846999c0e4dfab80ebe9d: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":249,"lbm_reads_lt_1ms":4}
I20260927 23:30:43.854229 6603 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:43.854655 6603 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:43.856439 6603 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:43.857863 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:30:43.858305 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:30:43.858585 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:43.859582 6609 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:43.889508 6603 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:43.889997 6603 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:43.891738 6603 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:43.892901 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:30:43.893460 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:30:43.893740 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:43.894289 6603 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:43.914340 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:44.213431 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:44.216672 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:44.334805 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:44.617827 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:44.793157 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 51: this message will repeat every 5th retry.
I20260927 23:30:45.686944 6622 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:45.687435 6622 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:45.689345 6622 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:45.691438 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:30:45.694046 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:30:45.694933 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:45.695889 6603 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:45.940199 6622 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:45.940665 6622 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:45.942215 6622 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:45.943645 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:30:45.944087 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:30:45.944403 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:45.945099 6622 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:45.964227 6622 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:45.964787 6622 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:45.966490 6622 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:45.968146 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:30:45.968600 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:30:45.968865 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:45.969388 6622 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:46.283219 6628 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:46.283797 6628 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:46.286022 6628 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:46.287573 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:30:46.288112 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:30:46.288462 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:46.289072 6628 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:46.432096 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 51: this message will repeat every 5th retry.
I20260927 23:30:46.471767 6603 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:46.472424 6603 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:46.474507 6603 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:46.476450 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:30:46.477238 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:30:46.477582 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:46.478271 6603 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:46.516335 6603 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:46.516956 6603 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:46.519446 6603 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:46.520941 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:30:46.521682 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:30:46.522046 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:46.522732 6603 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:46.581341 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:46.687544 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 51: this message will repeat every 5th retry.
W20260927 23:30:46.888796 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 56: this message will repeat every 5th retry.
I20260927 23:30:46.896756 6622 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:46.897307 6622 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:46.899315 6622 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:46.902136 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:30:46.902842 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:30:46.903178 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:46.903728 6622 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:47.493512 6603 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:47.494066 6603 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:47.496200 6603 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:47.497828 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:30:47.498253 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:30:47.498543 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:47.499148 6603 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:47.610464 6628 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:47.611094 6628 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:47.612919 6628 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:47.614382 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:30:47.615423 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:30:47.615696 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:47.616268 6628 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:47.715199 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 56: this message will repeat every 5th retry.
I20260927 23:30:47.755294 6603 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:47.755786 6603 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:47.757515 6603 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:47.759172 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:30:47.759789 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:30:47.760371 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:47.761420 6603 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:47.959161 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:48.070060 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:48.142567 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:48.173029 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:48.293397 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:48.384249 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 56: this message will repeat every 5th retry.
I20260927 23:30:48.629959 6642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:30:48.630549 6642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:30:48.633332 6642 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:30:48.640071 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:30:48.644194 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:30:48.644802 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:30:48.645251 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:30:48.645891 6642 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:30:49.100023 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:49.394025 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 51: this message will repeat every 5th retry.
I20260927 23:30:49.877269 6650 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:49.877722 6650 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:49.879689 6650 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:49.881278 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:30:49.881762 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:30:49.882141 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:49.882738 6650 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:50.197194 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 56: this message will repeat every 5th retry.
I20260927 23:30:50.381404 6637 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:50.381837 6637 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:30:50.383627 6637 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:30:50.384997 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:30:50.385442 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:30:50.385720 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:50.386363 6637 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:50.817397 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:51.151895 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:51.439611 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:51.703472 5164 heartbeater.cc:644] Failed to heartbeat to 127.2.57.61:46527 (2389 consecutive failures): Network error: Failed to ping master at 127.2.57.61:46527: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:30:51.874451 5020 heartbeater.cc:644] Failed to heartbeat to 127.2.57.61:46527 (202 consecutive failures): Network error: Failed to ping master at 127.2.57.61:46527: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:30:51.909669 6650 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:51.910337 6650 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:51.912663 6650 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:51.914281 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:30:51.914835 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:30:51.915086 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:51.915781 6650 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:52.477275 4879 heartbeater.cc:644] Failed to heartbeat to 127.2.57.61:46527 (204 consecutive failures): Network error: Failed to ping master at 127.2.57.61:46527: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:30:52.481951 6660 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:52.482460 6660 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:52.484453 6660 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:52.486110 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:30:52.486584 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:30:52.487103 5051 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:52.487703 6660 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:52.808010 6669 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:52.808495 6669 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:52.810164 6669 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:52.811627 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:30:52.812067 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:30:52.812384 5051 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:52.813031 6669 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:53.209314 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 56: this message will repeat every 5th retry.
I20260927 23:30:53.266338 6650 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:53.266988 6650 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:53.269500 6650 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:53.271065 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:30:53.271754 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:30:53.272107 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:53.272805 6650 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:30:53.348203 6650 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:53.348693 6650 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:30:53.350443 6650 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:30:53.351853 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:30:53.352537 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:30:53.352886 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:53.353544 6650 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:53.437724 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 56: this message will repeat every 5th retry.
I20260927 23:30:53.582499 6660 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:53.582964 6660 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:53.586176 6660 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:53.588840 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:30:53.589324 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:30:53.589692 5051 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:53.590262 6660 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:53.759728 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:54.079241 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:54.202836 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:54.480507 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:54.616323 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 56: this message will repeat every 5th retry.
I20260927 23:30:55.400823 6660 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:55.401351 6660 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:30:55.405633 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:30:55.406616 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:30:55.406804 6660 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:30:55.407006 5051 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:55.413160 6660 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:56.166329 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 56: this message will repeat every 5th retry.
I20260927 23:30:56.543646 6660 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:56.544168 6660 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:56.546953 6660 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:56.548571 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:30:56.549199 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:30:56.549585 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:56.550221 6660 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:56.641417 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 56: this message will repeat every 5th retry.
I20260927 23:30:56.692832 6660 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:56.693459 6660 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:56.695756 6660 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:56.696805 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:30:56.697373 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:30:56.697726 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:56.698352 6660 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:56.818413 6694 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:56.819067 6694 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:56.820766 6694 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:56.822091 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:30:56.822491 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:30:56.822813 5051 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:56.823415 6660 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:56.842444 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 56: this message will repeat every 5th retry.
W20260927 23:30:57.484544 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 61: this message will repeat every 5th retry.
W20260927 23:30:57.511339 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 61: this message will repeat every 5th retry.
W20260927 23:30:57.520009 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 61: this message will repeat every 5th retry.
W20260927 23:30:57.595481 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 61: this message will repeat every 5th retry.
W20260927 23:30:57.886062 5015 debug-util.cc:398] Leaking SignalData structure 0x7b08000c3100 after lost signal to thread 4889
W20260927 23:30:57.887116 5015 debug-util.cc:398] Leaking SignalData structure 0x7b0800043700 after lost signal to thread 5018
W20260927 23:30:57.937299 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 61: this message will repeat every 5th retry.
I20260927 23:30:58.235584 6669 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:58.236111 6669 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:58.238260 6669 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:58.239862 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:30:58.240355 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:30:58.240666 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:58.241237 6669 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:30:58.323101 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 61: this message will repeat every 5th retry.
I20260927 23:30:58.389889 6669 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:58.390416 6669 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:30:58.392107 6669 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:58.393561 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:30:58.394025 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:30:58.394347 5051 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:58.394985 6669 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:30:58.507220 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 61: this message will repeat every 5th retry.
W20260927 23:30:58.785681 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 61: this message will repeat every 5th retry.
W20260927 23:30:58.964529 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 61: this message will repeat every 5th retry.
W20260927 23:30:59.403571 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 56: this message will repeat every 5th retry.
I20260927 23:30:59.469976 6721 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:59.470463 6721 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:59.472119 6721 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:59.474138 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:30:59.474607 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:30:59.474995 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:59.475876 6721 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:30:59.923944 6718 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:30:59.924489 6718 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:30:59.926251 6718 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:30:59.928202 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:30:59.928620 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:30:59.928998 5051 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:30:59.929638 6718 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:00.177386 6669 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:00.177889 6669 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:00.179644 6669 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:00.180447 4615 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ac88a4e4bee64586bef7b6b409a63f4c consensus state has no leader
W20260927 23:31:00.181357 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:31:00.182020 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:31:00.182394 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:00.182976 6669 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:00.720788 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 61: this message will repeat every 5th retry.
I20260927 23:31:00.958226 6718 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:00.958932 6718 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:00.961076 6718 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:00.962939 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:31:00.963544 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:31:00.963804 5051 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:00.964462 6718 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:01.044025 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 61: this message will repeat every 5th retry.
W20260927 23:31:01.482381 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 61: this message will repeat every 5th retry.
I20260927 23:31:01.506186 6669 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:01.506757 6669 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:01.508878 6669 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:01.510471 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:31:01.510934 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:31:01.511394 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:01.512132 6669 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:01.571748 6669 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:01.572192 6669 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:01.573858 6669 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:01.575435 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:31:01.575834 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:31:01.576156 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:01.576746 6669 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:01.798985 6733 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:01.799494 6733 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:01.801839 6733 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:01.804999 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:31:01.806180 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:31:01.806603 5051 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:01.807377 6733 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:02.122372 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 61: this message will repeat every 5th retry.
I20260927 23:31:02.185166 6736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:31:02.185747 6736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:31:02.193269 6736 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:31:02.194954 4615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:31:02.198511 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:31:02.198983 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:31:02.199381 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:31:02.200224 6736 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:31:02.815703 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 61: this message will repeat every 5th retry.
W20260927 23:31:03.161060 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 61: this message will repeat every 5th retry.
I20260927 23:31:03.179147 6742 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:03.179788 6742 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:03.181664 6742 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:03.183439 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:31:03.183902 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:31:03.184185 5051 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:03.184808 6742 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:03.282897 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 61: this message will repeat every 5th retry.
I20260927 23:31:03.416333 6733 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:03.416810 6733 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:03.418529 6733 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:03.420068 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:31:03.420531 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:31:03.420814 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:03.421430 6733 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:03.911976 6727 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:03.912479 6727 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:03.914144 6727 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:03.915598 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:31:03.916036 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:31:03.916396 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:03.916947 6727 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:31:04.082309 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 61: this message will repeat every 5th retry.
I20260927 23:31:04.086591 6742 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:04.087162 6742 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:04.089217 6742 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:04.090566 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:31:04.091037 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:31:04.091331 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:04.091883 6742 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:04.319389 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 61: this message will repeat every 5th retry.
I20260927 23:31:04.372399 6727 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:04.372915 6727 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:04.374712 6727 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:04.376719 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:31:04.377463 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:31:04.377899 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:04.378726 6748 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:04.867694 6748 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:04.868356 6748 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:31:04.873234 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:31:04.873941 6748 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:04.874789 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:31:04.875159 5051 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:04.875900 6748 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:04.935304 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 61: this message will repeat every 5th retry.
W20260927 23:31:04.954173 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 61: this message will repeat every 5th retry.
I20260927 23:31:05.813758 6742 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:05.814375 6742 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:05.816624 6742 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:05.819046 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:31:05.819881 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:31:05.820258 5051 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:05.821384 6742 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:06.215710 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 327adcc2a27f42acb3a77d49fbf73796 consensus state has no leader
I20260927 23:31:06.335233 6748 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:06.335790 6748 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:06.337522 6748 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:06.339251 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:31:06.339668 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:31:06.339970 5051 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:06.340577 6748 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:06.662652 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 61: this message will repeat every 5th retry.
W20260927 23:31:06.788919 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 61: this message will repeat every 5th retry.
W20260927 23:31:06.953766 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 61: this message will repeat every 5th retry.
I20260927 23:31:07.128822 6748 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:07.129391 6748 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:07.131524 6748 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:07.133337 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:31:07.133800 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:31:07.134142 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:07.134691 6748 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:07.494239 4661 debug-util.cc:398] Leaking SignalData structure 0x7b080004ff80 after lost signal to thread 4593
W20260927 23:31:07.495087 4661 debug-util.cc:398] Leaking SignalData structure 0x7b080009c300 after lost signal to thread 4664
W20260927 23:31:07.666554 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 66: this message will repeat every 5th retry.
I20260927 23:31:07.735157 6748 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:07.735630 6748 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:31:07.737342 6748 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:31:07.738965 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:31:07.739318 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:31:07.739631 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:07.740342 6748 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:07.766649 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:07.771615 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:07.786057 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:07.881644 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 1766d227273e4c7592c4f0a20f2f7b17 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1766d227273e4c7592c4f0a20f2f7b17 consensus state has no leader
W20260927 23:31:07.921779 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:08.125475 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 66: this message will repeat every 5th retry.
I20260927 23:31:08.355086 6764 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:08.355681 6764 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:08.358003 6764 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:08.359726 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:31:08.360275 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:31:08.360669 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:08.361322 6764 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:08.438853 6764 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:08.439393 6764 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:08.441761 6764 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:08.442787 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:31:08.443264 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:31:08.443629 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:08.444281 6764 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:08.452639 6764 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:08.453102 6764 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:08.454784 6764 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:08.456255 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:31:08.456727 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:31:08.457114 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:08.457690 6764 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:08.615105 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:08.896071 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 66: this message will repeat every 5th retry.
I20260927 23:31:08.928049 6764 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:08.928637 6764 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:31:08.930830 6764 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:31:08.934330 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:31:08.936285 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:31:08.936923 5054 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:08.938023 6768 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:09.027602 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 66: this message will repeat every 5th retry.
I20260927 23:31:09.561502 6764 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:09.562074 6764 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:09.564036 6764 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:09.565716 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:31:09.566669 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:31:09.567077 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:09.568274 6764 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:09.833325 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 61: this message will repeat every 5th retry.
I20260927 23:31:10.136790 6764 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:10.137307 6764 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:10.139282 6764 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:10.141451 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:31:10.143258 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:31:10.143574 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:10.144160 6727 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:10.261875 6727 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:10.262421 6727 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:10.263985 6727 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:10.265667 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:31:10.266166 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:31:10.266489 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:10.267211 6727 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:31:10.509765 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:10.867660 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9f07e8a37404636bda765146b20fc60 consensus state has no leader
W20260927 23:31:11.421196 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:11.515827 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:11.912675 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:12.510001 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 66: this message will repeat every 5th retry.
I20260927 23:31:12.695052 6774 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:12.695760 6774 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:12.698021 6774 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:12.699741 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:31:12.700179 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:31:12.700475 5051 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:12.701076 6774 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:12.763782 6774 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:12.764263 6774 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:12.766162 6774 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:12.767530 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:31:12.768126 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:31:12.768540 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:12.769404 6774 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:12.974747 4661 debug-util.cc:398] Leaking SignalData structure 0x7b08000a1c80 after lost signal to thread 4593
W20260927 23:31:12.976094 4661 debug-util.cc:398] Leaking SignalData structure 0x7b080004ef00 after lost signal to thread 4664
W20260927 23:31:12.989809 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 66: this message will repeat every 5th retry.
I20260927 23:31:13.567273 6774 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:13.567806 6774 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:13.569871 6774 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:13.571472 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:31:13.571875 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:31:13.572261 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:13.573032 6774 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:13.629166 6774 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:13.630338 6774 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:13.635361 6774 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:13.636454 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:31:13.637249 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:31:13.637532 5051 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:13.638093 6774 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:13.950649 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:14.071364 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:14.573659 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:14.594049 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 66: this message will repeat every 5th retry.
I20260927 23:31:14.630702 6783 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:31:14.631562 4874 debug-util.cc:398] Leaking SignalData structure 0x7b0800115800 after lost signal to thread 4744
I20260927 23:31:14.631271 6783 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:31:14.632992 4874 debug-util.cc:398] Leaking SignalData structure 0x7b0800082ae0 after lost signal to thread 4877
I20260927 23:31:14.634459 6783 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:14.635890 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:31:14.636492 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:31:14.636886 5051 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:14.638024 6774 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:14.978750 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:16.288092 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 66: this message will repeat every 5th retry.
W20260927 23:31:16.349241 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 66: this message will repeat every 5th retry.
I20260927 23:31:17.095937 6798 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:17.096441 6798 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:17.098140 6798 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:17.100000 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:31:17.100504 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:31:17.100807 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:17.101475 6798 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:17.196882 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 66: this message will repeat every 5th retry.
I20260927 23:31:17.327637 6778 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:17.328159 6778 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:17.330221 6778 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:17.332510 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:31:17.334054 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:31:17.334578 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:17.335184 6798 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:17.335309 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:17.393877 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:17.871047 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:17.915769 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 71: this message will repeat every 5th retry.
I20260927 23:31:17.933816 6798 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:17.934274 6798 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:17.936012 6798 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:17.937631 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:31:17.938028 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:31:17.938338 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:17.939276 6798 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:18.478971 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:19.073163 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:19.116261 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:19.262882 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 71: this message will repeat every 5th retry.
I20260927 23:31:19.312145 6805 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:19.312697 6805 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:19.314527 6805 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:19.316866 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:31:19.317747 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:31:19.318151 5051 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:19.318909 6805 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:19.327526 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 71: this message will repeat every 5th retry.
I20260927 23:31:19.561323 6815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:31:19.562044 6815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:31:19.564599 6815 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:31:19.566982 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:31:19.570228 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:31:19.570791 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:31:19.571085 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:31:19.571631 6815 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:31:19.663813 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 66: this message will repeat every 5th retry.
I20260927 23:31:20.101645 6823 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:20.102550 6823 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:31:20.106731 6823 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:31:20.107815 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:31:20.108222 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:31:20.108632 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:20.109357 6808 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:20.160686 6808 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:20.161298 6808 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:20.162993 6808 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:20.164464 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:31:20.165040 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:31:20.165334 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:20.165942 6808 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:20.414513 6808 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:20.415059 6808 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:20.416860 6808 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:20.418548 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:31:20.418951 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:31:20.419219 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:20.419799 6808 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:20.450567 6808 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:20.451058 6808 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:31:20.452750 6808 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:31:20.454273 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:31:20.454630 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:31:20.454916 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:20.455503 6808 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:20.586056 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:20.586735 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 71: this message will repeat every 5th retry.
I20260927 23:31:20.618619 6808 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:20.619171 6808 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:20.621069 6808 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:20.622568 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:31:20.623270 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:31:20.623564 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:20.624169 6808 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:21.487318 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:21.873946 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:22.681896 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:22.699549 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:23.195571 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 71: this message will repeat every 5th retry.
I20260927 23:31:23.394230 6851 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:23.394804 6851 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:23.396863 6851 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:23.398586 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:31:23.399016 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:31:23.399302 5051 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:23.399930 6851 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:23.608234 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 71: this message will repeat every 5th retry.
I20260927 23:31:23.625872 6850 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:23.626299 6850 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:23.627975 6850 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:23.629539 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:31:23.629987 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:31:23.630288 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:23.630970 6850 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:23.888453 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 71: this message will repeat every 5th retry.
I20260927 23:31:24.381213 6865 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:31:24.381778 6865 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:31:24.383622 6865 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:31:24.389945 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:31:24.390713 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:31:24.391039 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:31:24.391621 6865 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:31:24.664074 6805 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:24.664665 6805 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:24.666579 6805 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:24.668018 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:31:24.668515 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:31:24.668943 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:24.669652 6805 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:24.855154 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:24.965909 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 71: this message will repeat every 5th retry.
I20260927 23:31:24.966599 6871 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:24.967103 6871 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:24.969561 6871 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:24.973335 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:31:24.975201 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:31:24.975602 5051 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:24.976372 6871 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:25.071143 6860 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:25.071631 6860 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:25.073453 6860 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:25.075031 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:31:25.075551 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:31:25.075851 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:25.076607 6860 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:31:25.299849 6877 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:31:25.301867 6877 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:31:25.311242 6877 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:31:25.312353 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260927 23:31:25.315322 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:31:25.315681 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:31:25.315992 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:31:25.316653 6877 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:31:25.848265 6871 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:25.848793 6871 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:25.850495 6871 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:25.852157 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:31:25.852586 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:31:25.852931 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:25.853467 6871 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:25.911379 6850 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:25.911854 6850 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:25.913457 6850 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:25.915036 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:31:25.915493 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:31:25.915776 5051 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:25.916328 6850 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:26.247236 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 71: this message will repeat every 5th retry.
I20260927 23:31:26.254181 6850 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:26.254760 6850 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:26.256801 6850 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:26.258111 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:31:26.258987 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:31:26.259361 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:26.260100 6850 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:26.278530 6850 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:26.279271 6850 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:26.281512 6850 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:26.283133 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:31:26.283582 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:31:26.283882 5051 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:26.284499 6850 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:26.464071 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:26.821594 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:27.143783 6805 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:27.144397 6805 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:27.146276 6805 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:27.148190 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:31:27.148604 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:31:27.148900 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:27.149456 6805 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:27.261181 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 71: this message will repeat every 5th retry.
I20260927 23:31:28.019462 6805 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:28.019969 6805 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:28.021799 6805 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:28.023630 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:31:28.024361 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:31:28.024762 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:28.025542 6805 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:28.061975 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 76: this message will repeat every 5th retry.
W20260927 23:31:28.101883 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:28.115722 6805 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:28.116346 6805 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:28.117954 6805 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:28.119576 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:31:28.119990 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:31:28.120262 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:28.120803 6805 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:28.857132 6805 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:28.857590 6805 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:28.859056 6805 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:28.860690 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:31:28.861083 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:31:28.861368 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:28.861903 6805 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:28.890460 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 76: this message will repeat every 5th retry.
W20260927 23:31:28.914162 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 76: this message will repeat every 5th retry.
W20260927 23:31:29.132979 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:29.261318 6871 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:31:29.261992 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:29.261852 6871 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:29.263818 6871 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:29.266044 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:31:29.266563 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:31:29.266928 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:29.267545 6871 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:29.324056 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 76: this message will repeat every 5th retry.
W20260927 23:31:29.434038 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:29.861135 6871 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:29.861640 6871 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:31:29.863345 6871 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:31:29.865329 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:31:29.865772 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:31:29.866118 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:29.866681 6871 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:29.925853 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 76: this message will repeat every 5th retry.
W20260927 23:31:30.052664 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 71: this message will repeat every 5th retry.
W20260927 23:31:30.291409 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:30.698132 6890 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:30.698648 6890 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:30.700805 6890 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:30.702533 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:31:30.702994 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:31:30.703317 5051 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:30.703981 6890 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:31.170464 6805 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:31.171221 6805 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
W20260927 23:31:31.176467 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:31:31.179731 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:31:31.179731 6805 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
I20260927 23:31:31.180246 5051 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:31.181087 6805 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:31.674062 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 76: this message will repeat every 5th retry.
W20260927 23:31:31.902495 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 76: this message will repeat every 5th retry.
W20260927 23:31:32.156458 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 76: this message will repeat every 5th retry.
W20260927 23:31:32.812225 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:32.846201 6805 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:32.846694 6805 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:32.848249 6805 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:32.849918 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:31:32.850310 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:31:32.850675 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:32.851394 6805 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:32.943122 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 76: this message will repeat every 5th retry.
W20260927 23:31:33.805276 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:33.953322 6871 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:33.953816 6871 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:33.955470 6871 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:33.957108 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:31:33.957504 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:31:33.957796 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:33.958362 6871 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:34.073321 6871 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:34.073863 6871 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:34.075320 6871 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:34.076980 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:31:34.077409 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:31:34.077703 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:34.078404 6871 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:34.101274 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:34.236240 6871 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:34.236836 6871 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:34.238686 6871 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:34.240334 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:31:34.240782 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:31:34.241070 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:34.241621 6871 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:34.328465 6871 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:34.328874 6871 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:34.330283 6871 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:34.331856 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:31:34.332252 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:31:34.332669 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:34.333353 6871 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:34.385825 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 76: this message will repeat every 5th retry.
W20260927 23:31:34.578823 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:35.040704 6871 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:35.041240 6871 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:35.042845 6871 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:35.044520 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:31:35.044907 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:31:35.045292 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:35.046016 6871 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:35.613485 6805 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:35.613963 6805 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:31:35.615679 6805 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:31:35.617446 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:31:35.617954 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:31:35.618248 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:35.618921 6805 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:35.763669 6805 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:35.764183 6805 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:35.766252 6805 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:35.768814 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:31:35.769479 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:31:35.769858 5051 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:35.770920 6805 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:36.052018 6890 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:31:36.052495 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:36.052510 6890 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:36.055126 6890 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:36.056468 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:31:36.057402 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:31:36.057785 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:36.058485 6890 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:36.359256 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:36.409201 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:37.185070 6953 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:37.185633 6953 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:31:37.189596 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:31:37.189108 6953 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:37.190420 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:31:37.190856 6953 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:37.190810 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:37.191505 6890 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:37.191258 6953 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:37.193192 6953 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:37.194664 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:31:37.195204 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:31:37.195611 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:37.196370 6805 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:37.570606 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 81: this message will repeat every 5th retry.
I20260927 23:31:37.669914 6890 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:37.670452 6890 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:31:37.672560 6890 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:31:37.674048 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:31:37.674474 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:31:37.674805 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:37.675365 6890 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:38.316200 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:38.391731 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 76: this message will repeat every 5th retry.
W20260927 23:31:38.814257 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:38.854506 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 81: this message will repeat every 5th retry.
I20260927 23:31:38.958004 6890 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:38.958438 6890 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:38.960340 6890 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:38.962025 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:31:38.962478 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:31:38.962859 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:38.963615 6890 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:38.983577 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 81: this message will repeat every 5th retry.
I20260927 23:31:39.011888 6805 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:39.012552 6805 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:39.014430 6805 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:39.016268 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:31:39.016767 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:31:39.017151 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:39.017791 6805 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:39.572588 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:39.764111 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:40.105876 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:40.394182 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:40.507781 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:40.625790 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 76: this message will repeat every 5th retry.
I20260927 23:31:40.941601 6989 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:40.942518 6989 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:40.945111 6989 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:40.946544 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:31:40.947175 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:31:40.947497 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:40.948176 6989 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:31:41.468341 6988 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:41.468822 6988 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:41.470737 6988 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:41.472563 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:31:41.473079 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:31:41.473382 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:41.473965 6988 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:41.678119 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:41.931936 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:42.014469 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 81: this message will repeat every 5th retry.
I20260927 23:31:42.241016 6998 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:42.241658 6998 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:42.243778 6998 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:42.245811 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:31:42.248651 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:31:42.249090 5051 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:42.249943 6999 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:42.312361 6999 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:42.312821 6999 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:42.314648 6999 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:42.315948 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:31:42.316608 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:31:42.316937 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:42.317480 6999 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:42.657637 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 81: this message will repeat every 5th retry.
I20260927 23:31:42.686142 6988 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:42.686864 6988 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:31:42.689546 6988 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:31:42.690842 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:31:42.691350 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:31:42.691660 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:42.692286 6988 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:43.078339 7017 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:31:43.079000 7017 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:31:43.081643 7017 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:31:43.084350 4615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111) [suppressed 2 similar messages]
W20260927 23:31:43.087145 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:31:43.087649 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:31:43.087952 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:31:43.088582 7017 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:31:43.117077 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 81: this message will repeat every 5th retry.
I20260927 23:31:43.487938 6999 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:43.488502 6999 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:43.491063 6999 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:43.492434 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:31:43.492882 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:31:43.493193 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:43.493865 6999 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:43.813028 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 81: this message will repeat every 5th retry.
I20260927 23:31:43.814172 6999 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:43.814728 6999 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:43.816932 6999 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:43.818420 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:31:43.818926 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:31:43.819227 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:43.819808 6999 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:43.867728 7023 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:43.868340 7023 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:43.870119 7023 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:43.871574 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:31:43.872045 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:31:43.872398 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:43.872961 7023 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:43.920760 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:44.388947 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 81: this message will repeat every 5th retry.
I20260927 23:31:44.752225 7036 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:44.752807 7036 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:44.754925 7036 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:44.756688 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:31:44.757164 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:31:44.757539 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:44.758175 7036 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:44.894049 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:45.803220 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:46.026895 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 81: this message will repeat every 5th retry.
I20260927 23:31:46.101790 6998 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:46.102352 6998 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:46.103964 6998 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:46.105671 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:31:46.106516 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:31:46.106848 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:46.107515 6998 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:46.668918 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 86: this message will repeat every 5th retry.
I20260927 23:31:46.782332 7046 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:46.783228 7046 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:46.785101 7046 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:46.786605 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:31:46.787048 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:31:46.787555 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:46.788130 6998 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:46.973048 7036 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:46.973564 7036 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:46.975875 7036 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:46.977604 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:31:46.978139 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:31:46.978478 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:46.979086 7036 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:31:47.180341 6998 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:47.180860 6998 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:31:47.182920 6998 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:31:47.184547 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:31:47.185088 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:31:47.185402 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:47.185966 6998 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:47.233196 6998 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:47.233738 6998 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:47.235916 6998 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:47.237310 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:31:47.237773 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:31:47.238131 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:47.238859 6998 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:47.507390 7036 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:47.507944 7036 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:47.509800 7036 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:47.511613 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:31:47.512100 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:31:47.512436 5051 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:47.513068 7036 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:48.068142 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:48.141219 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:48.370878 4661 debug-util.cc:398] Leaking SignalData structure 0x7b08000b98e0 after lost signal to thread 4593
W20260927 23:31:48.371517 4661 debug-util.cc:398] Leaking SignalData structure 0x7b08000bc400 after lost signal to thread 4664
W20260927 23:31:48.430085 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 86: this message will repeat every 5th retry.
I20260927 23:31:48.695596 7057 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:48.696130 7057 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:48.698163 7057 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:48.699890 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:31:48.700546 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:31:48.700857 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:48.701665 7057 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:48.994876 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:49.143748 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:49.236090 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:49.371685 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:50.080969 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:50.212208 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 86: this message will repeat every 5th retry.
I20260927 23:31:50.514799 7069 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:50.515360 7069 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:50.517297 7069 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:50.518936 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:31:50.519339 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:31:50.519652 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:50.520215 7069 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:50.555902 7069 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:50.556523 7069 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:50.558404 7069 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:50.560041 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:31:50.560536 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:31:50.560817 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:50.561376 7069 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:50.731070 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:51.123224 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 81: this message will repeat every 5th retry.
W20260927 23:31:51.140533 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:51.445046 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:51.519620 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:51.706374 5165 heartbeater.cc:644] Failed to heartbeat to 127.2.57.62:35155 (3779 consecutive failures): Network error: Failed to ping master at 127.2.57.62:35155: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:31:51.735920 7069 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:51.736555 7069 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:51.738503 7069 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:51.740207 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:31:51.740839 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:31:51.741125 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:51.741667 7069 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:52.002085 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:52.364830 5020 heartbeater.cc:644] Failed to heartbeat to 127.2.57.61:46527 (322 consecutive failures): Network error: Failed to ping master at 127.2.57.61:46527: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:31:52.406682 5159 debug-util.cc:398] Leaking SignalData structure 0x7b08000d7560 after lost signal to thread 5032
W20260927 23:31:52.408128 5159 debug-util.cc:398] Leaking SignalData structure 0x7b0800116620 after lost signal to thread 5162
W20260927 23:31:52.608031 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:52.782841 4879 heartbeater.cc:644] Failed to heartbeat to 127.2.57.61:46527 (324 consecutive failures): Network error: Failed to ping master at 127.2.57.61:46527: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:31:53.175400 7076 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:53.175972 7076 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:53.177884 7076 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:53.179378 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:31:53.179803 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:31:53.180117 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:53.180771 7076 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:53.419094 7102 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:53.419502 7102 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:53.421569 7102 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:53.423733 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:31:53.424175 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:31:53.424526 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:53.425064 7102 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:53.745095 7102 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:53.745676 7102 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:31:53.747578 7102 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:31:53.749130 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:31:53.749681 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:31:53.749970 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:53.750924 7102 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:53.894658 7108 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:53.895138 7108 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:53.897086 7108 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:53.898242 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:31:53.899165 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:31:53.899588 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:53.901077 7102 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:54.060575 7076 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:54.061211 7076 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:31:54.062904 7076 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:31:54.064496 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:31:54.064877 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:31:54.065150 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:54.065667 7076 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:54.090152 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:54.137077 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 86: this message will repeat every 5th retry.
I20260927 23:31:54.197156 7076 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:54.197659 7076 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:54.199383 7076 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:54.200955 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:31:54.201654 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:31:54.202026 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:54.202756 7076 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:54.289386 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:54.393563 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:54.457176 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 86: this message will repeat every 5th retry.
I20260927 23:31:54.576243 7108 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:54.576759 7108 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:54.578675 7108 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:54.580158 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:31:54.580608 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:31:54.580905 5051 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:54.581449 7108 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:54.728711 7108 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:54.729244 7108 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:54.731477 7108 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:54.734100 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:31:54.735875 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:31:54.736284 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:54.737685 7069 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:54.825011 7069 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:54.825511 7069 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:54.827205 7069 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:54.828994 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:31:54.829463 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:31:54.829828 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:54.830502 7069 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:54.957139 7108 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:54.957722 7108 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:54.959702 7108 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:54.961216 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:31:54.961655 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:31:54.961968 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:54.962556 7108 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:55.361460 7113 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:55.362010 7113 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:55.363965 7113 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:55.365640 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:31:55.366065 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:31:55.366323 5051 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:55.366858 7113 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:31:55.462050 7076 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:55.462548 7076 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:55.464566 7076 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:55.466010 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:31:55.466451 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:31:55.466784 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:55.467487 7076 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:31:55.605937 7113 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:55.606405 7113 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:55.608150 7113 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:55.609834 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:31:55.610232 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:31:55.610594 5051 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:55.611191 7113 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:55.653721 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:56.383703 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:56.724073 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:31:57.657210 7076 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:57.657779 7076 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:57.659683 7076 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:57.661319 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:31:57.661757 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:31:57.662034 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:57.662650 7076 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:57.687479 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:31:57.711494 7129 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:31:57.712158 7129 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:31:57.714124 7129 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:31:57.716759 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:31:57.720773 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:31:57.721448 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:31:57.721762 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:31:57.722636 7129 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:31:57.965193 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:31:58.198647 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 91: this message will repeat every 5th retry.
W20260927 23:31:58.281328 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 91: this message will repeat every 5th retry.
W20260927 23:31:58.284143 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:31:58.496930 7136 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:58.497435 7136 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:58.499544 7136 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:58.501178 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:31:58.501650 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:31:58.501948 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:58.502506 7136 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:58.815853 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:31:58.851075 7143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:31:58.851670 7143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:31:58.853971 7143 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
I20260927 23:31:58.856963 7142 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:58.857497 7142 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:31:58.859684 7142 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:58.859670 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:31:58.860486 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:31:58.860868 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
W20260927 23:31:58.861358 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:31:58.861573 7143 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:31:58.862754 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:31:58.863085 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:58.863692 7142 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:59.179075 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:31:59.812636 7122 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:31:59.813074 7122 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:31:59.814888 7122 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:31:59.816505 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:31:59.817320 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:31:59.817802 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:31:59.818485 7122 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:31:59.879252 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 91: this message will repeat every 5th retry.
W20260927 23:31:59.923425 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:32:00.011267 7142 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:00.011766 7142 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:00.013815 7142 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:00.015800 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:32:00.016752 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:32:00.018793 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:00.021512 7122 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:00.094287 7122 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:00.094726 7122 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:00.096262 7122 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:00.097647 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:32:00.098035 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:32:00.098300 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:00.098821 7122 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:00.226979 4615 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ac88a4e4bee64586bef7b6b409a63f4c consensus state has no leader
W20260927 23:32:00.551469 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 86: this message will repeat every 5th retry.
W20260927 23:32:00.575655 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:32:01.006398 7155 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:01.007172 7155 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:01.009660 7155 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:01.011333 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:32:01.012063 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:32:01.012513 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:01.013909 7149 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:32:01.212396 7149 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:01.212955 7149 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:01.214720 7149 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:01.216354 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:32:01.216760 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:32:01.217092 5051 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:01.217733 7149 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:01.486819 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 91: this message will repeat every 5th retry.
W20260927 23:32:01.604036 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:32:01.861991 7163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:32:01.862468 7163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:32:01.864255 7163 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:32:01.869529 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:32:01.870432 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:32:01.870824 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:32:01.871383 7163 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:32:01.904909 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 91: this message will repeat every 5th retry.
W20260927 23:32:01.948084 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 91: this message will repeat every 5th retry.
W20260927 23:32:02.687172 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:32:03.787693 7167 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:03.788157 7167 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:03.790071 7167 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:03.793042 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:32:03.793852 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:32:03.794236 5051 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:03.794903 7167 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:04.371255 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 91: this message will repeat every 5th retry.
W20260927 23:32:04.459337 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 91: this message will repeat every 5th retry.
W20260927 23:32:04.633203 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 91: this message will repeat every 5th retry.
W20260927 23:32:04.710951 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:32:04.794868 7167 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:04.795346 7167 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:04.797161 7167 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:04.798585 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:32:04.799531 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:32:04.799822 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:04.800390 7167 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:04.971436 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:32:05.167295 7189 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:05.167754 7189 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:05.169822 7189 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:05.171336 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:32:05.171772 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:32:05.172196 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:05.173043 7189 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:06.181118 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 91: this message will repeat every 5th retry.
W20260927 23:32:06.216158 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:32:06.221248 7184 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:06.221711 7184 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:06.223524 7184 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:06.226703 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:32:06.227334 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:32:06.227811 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:06.228498 7184 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:06.250948 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 327adcc2a27f42acb3a77d49fbf73796 consensus state has no leader
I20260927 23:32:06.468037 7200 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:06.468629 7200 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:06.470757 7200 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:06.472357 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:32:06.472779 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:32:06.473135 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:06.473850 7200 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:06.713441 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 96: this message will repeat every 5th retry.
I20260927 23:32:06.787797 7184 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:06.788287 7184 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:06.790308 7184 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:06.791571 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:32:06.792238 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:32:06.792573 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:06.793128 7184 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:07.005169 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:32:07.093685 7184 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:07.094221 7184 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:07.095988 7184 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:07.097617 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:32:07.098013 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:32:07.098367 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:07.098960 7184 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:07.160375 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 96: this message will repeat every 5th retry.
W20260927 23:32:07.442741 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 96: this message will repeat every 5th retry.
I20260927 23:32:07.755321 7200 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:07.755841 7200 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:07.757787 7200 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:07.759572 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:32:07.760032 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:32:07.760382 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:07.760932 7200 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:07.781661 7200 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:07.782152 7200 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:07.783798 7200 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:07.785490 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:32:07.785888 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:32:07.786221 5051 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:07.786980 7200 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:07.910949 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 1766d227273e4c7592c4f0a20f2f7b17 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1766d227273e4c7592c4f0a20f2f7b17 consensus state has no leader
W20260927 23:32:08.602746 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 96: this message will repeat every 5th retry.
W20260927 23:32:08.673199 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 96: this message will repeat every 5th retry.
I20260927 23:32:08.878423 7142 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:08.878980 7142 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:08.880910 7142 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:08.882655 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:32:08.883085 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:32:08.883404 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:08.883939 7142 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:09.008826 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 96: this message will repeat every 5th retry.
W20260927 23:32:09.428419 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 96: this message will repeat every 5th retry.
W20260927 23:32:09.604874 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 96: this message will repeat every 5th retry.
W20260927 23:32:09.618391 5015 debug-util.cc:398] Leaking SignalData structure 0x7b0800120a60 after lost signal to thread 4889
W20260927 23:32:09.620070 5015 debug-util.cc:398] Leaking SignalData structure 0x7b08000f9cc0 after lost signal to thread 5018
I20260927 23:32:09.724483 7210 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:09.725031 7210 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:32:09.727463 7210 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:32:09.729367 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:32:09.731271 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:32:09.731724 5054 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:09.732568 7210 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:09.800843 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 96: this message will repeat every 5th retry.
W20260927 23:32:09.965840 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 96: this message will repeat every 5th retry.
I20260927 23:32:09.992365 7218 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:09.992877 7218 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:09.994589 7218 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:09.996155 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:32:09.996621 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:32:09.996932 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:09.997498 7218 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:10.157061 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 91: this message will repeat every 5th retry.
I20260927 23:32:10.334543 7200 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:10.335146 7200 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:10.337024 7200 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:10.338685 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:32:10.339123 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:32:10.339458 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:10.340075 7200 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:10.489533 7224 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:10.490211 7224 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:32:10.494508 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:32:10.495493 7224 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:10.497174 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:32:10.497596 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:10.498306 7224 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:10.707499 7200 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:10.708079 7200 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:32:10.711030 7200 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:32:10.712021 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:32:10.712731 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:32:10.713124 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:10.714064 7200 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:10.869275 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9f07e8a37404636bda765146b20fc60 consensus state has no leader
I20260927 23:32:11.020078 7218 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:11.020947 7218 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:11.023756 7218 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:11.025344 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:32:11.026777 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:32:11.027211 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:11.027921 7218 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:11.303195 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 96: this message will repeat every 5th retry.
I20260927 23:32:11.670432 7200 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:11.671077 7200 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:11.673363 7200 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:11.675254 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:32:11.675704 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:32:11.675983 5051 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:11.676743 7200 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:11.865526 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 96: this message will repeat every 5th retry.
I20260927 23:32:11.882258 7200 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:11.883023 7200 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:11.885121 7200 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:11.886423 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:32:11.886893 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:32:11.887306 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:11.888015 7200 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:11.908691 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 96: this message will repeat every 5th retry.
W20260927 23:32:12.020733 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 96: this message will repeat every 5th retry.
W20260927 23:32:12.557456 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 96: this message will repeat every 5th retry.
I20260927 23:32:12.707916 7200 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:12.708541 7200 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:12.710681 7200 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:12.712492 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:32:12.713028 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:32:12.713403 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:12.714119 7200 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:12.962963 7224 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:12.963377 7224 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:12.965189 7224 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:12.966765 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:32:12.967216 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:32:12.967516 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:12.968195 7224 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:13.123153 7200 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:13.123677 7200 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:13.125519 7200 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:13.127189 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:32:13.128286 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:32:13.128726 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:13.129287 7200 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:14.510701 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 96: this message will repeat every 5th retry.
W20260927 23:32:14.626473 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 96: this message will repeat every 5th retry.
I20260927 23:32:14.680948 7200 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:14.681356 7200 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:14.683247 7200 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:14.684983 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:32:14.686196 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:32:14.686509 5051 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:14.687098 7200 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:14.701884 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 96: this message will repeat every 5th retry.
W20260927 23:32:14.751230 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 96: this message will repeat every 5th retry.
W20260927 23:32:15.014609 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 96: this message will repeat every 5th retry.
I20260927 23:32:15.559803 7273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:32:15.560456 7273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:32:15.562557 7273 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:32:15.571383 4615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260927 23:32:15.575446 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:32:15.576174 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:32:15.576555 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:32:15.577122 7273 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:32:15.600761 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 96: this message will repeat every 5th retry.
I20260927 23:32:15.930576 7224 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:15.931064 7224 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:15.932951 7224 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:15.934638 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:32:15.935071 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:32:15.935351 5051 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:15.935917 7224 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:15.940587 7224 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:15.941128 7224 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:15.942655 7224 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:15.944083 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:32:15.944483 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:32:15.944790 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:15.945391 7224 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:16.500032 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:16.596020 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:16.739640 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 96: this message will repeat every 5th retry.
W20260927 23:32:17.064131 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 101: this message will repeat every 5th retry.
I20260927 23:32:17.104204 7200 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:17.104698 7200 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:17.106335 7200 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:17.108076 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:32:17.108502 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:32:17.108786 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:17.109368 7200 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:17.124188 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 96: this message will repeat every 5th retry.
I20260927 23:32:17.275439 7200 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:17.275893 7200 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:17.277411 7200 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:17.279130 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:32:17.279650 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:32:17.279970 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:17.280558 7200 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:18.010581 7289 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:18.011039 7289 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:18.012754 7289 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:18.014582 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:32:18.014987 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:32:18.015295 5051 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:18.015873 7289 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:18.081321 7289 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:18.081826 7289 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:18.083561 7289 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:18.085063 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:32:18.085567 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:32:18.085880 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:18.086440 7289 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:32:18.355193 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 101: this message will repeat every 5th retry.
I20260927 23:32:18.546638 7267 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:18.547246 7267 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:18.549487 7267 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:18.551534 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:32:18.552078 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:32:18.552493 5051 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:18.553153 7267 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:18.778294 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:18.996215 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:19.005733 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 101: this message will repeat every 5th retry.
I20260927 23:32:19.452852 7298 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:19.453375 7298 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:19.455389 7298 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:19.456959 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:32:19.457355 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:32:19.457628 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:19.458199 7298 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:19.470434 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 101: this message will repeat every 5th retry.
I20260927 23:32:19.524673 7298 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:19.525266 7298 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:19.527614 7298 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:19.529289 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:32:19.530184 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:32:19.530596 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:19.531298 7298 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:19.768920 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:19.926867 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:20.474382 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 96: this message will repeat every 5th retry.
I20260927 23:32:21.194257 7281 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:21.195181 7281 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:21.197641 7281 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:21.199343 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:32:21.200054 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:32:21.200423 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:21.201161 7281 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:21.419373 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:21.469053 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 101: this message will repeat every 5th retry.
I20260927 23:32:21.686928 7317 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:21.687538 7317 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:21.690061 7317 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:21.691009 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:32:21.692167 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:32:21.692580 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:21.693426 7317 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:21.728199 7307 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:21.728709 7307 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:21.730834 7307 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:21.733845 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:32:21.735802 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:32:21.736330 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:21.737116 7307 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:21.870711 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 101: this message will repeat every 5th retry.
I20260927 23:32:22.049226 7307 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:22.049769 7307 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:22.051455 7307 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:22.053064 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:32:22.053484 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:32:22.053789 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:22.054382 7307 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:22.315359 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 101: this message will repeat every 5th retry.
I20260927 23:32:22.460433 7320 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:22.460994 7320 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:22.463449 7320 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:22.465013 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:32:22.465430 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:32:22.465816 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:22.466563 7320 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:22.689287 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 101: this message will repeat every 5th retry.
I20260927 23:32:22.805588 7307 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:22.806031 7307 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:22.808086 7307 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:22.809927 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:32:22.812122 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:32:22.812565 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:22.813236 7307 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:23.921998 7317 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:23.922556 7317 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:23.924355 7317 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:23.925948 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:32:23.927000 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:32:23.927353 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:23.928088 7317 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:24.003403 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:24.034134 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:24.295666 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:24.864182 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:25.615604 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:25.654910 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 101: this message will repeat every 5th retry.
I20260927 23:32:25.783627 7342 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:25.784128 7342 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:25.785864 7342 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:25.787562 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:32:25.787918 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:32:25.788218 5051 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:25.788915 7342 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:25.840248 7342 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:25.840765 7342 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:25.842788 7342 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:25.844547 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:32:25.844985 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:32:25.845314 5051 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:25.845952 7342 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:26.422272 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 101: this message will repeat every 5th retry.
I20260927 23:32:26.604723 7343 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:26.605247 7343 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:32:26.607048 7343 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:32:26.608623 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:32:26.609102 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:32:26.609406 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:26.610013 7343 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:26.702291 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 106: this message will repeat every 5th retry.
W20260927 23:32:26.976994 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 101: this message will repeat every 5th retry.
W20260927 23:32:27.020202 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 106: this message will repeat every 5th retry.
W20260927 23:32:27.232956 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 106: this message will repeat every 5th retry.
W20260927 23:32:27.255824 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 106: this message will repeat every 5th retry.
I20260927 23:32:27.875356 7355 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:27.875829 7355 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:27.877573 7355 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:27.879364 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:32:27.879737 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:32:27.880012 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:27.880694 7355 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:28.426783 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 106: this message will repeat every 5th retry.
W20260927 23:32:29.043118 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 106: this message will repeat every 5th retry.
I20260927 23:32:29.074234 7342 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:29.074754 7342 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:29.076453 7342 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:29.078092 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:32:29.078514 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:32:29.078814 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:29.079336 7342 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:29.186456 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 106: this message will repeat every 5th retry.
I20260927 23:32:29.336337 7373 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:29.336853 7373 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:29.338804 7373 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:29.340427 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:32:29.340927 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:32:29.341214 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:29.342079 7373 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:29.376441 7373 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:29.376961 7373 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
W20260927 23:32:29.380726 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:32:29.379962 7373 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:32:29.381920 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:32:29.382357 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:29.383406 7373 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:29.507197 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 106: this message will repeat every 5th retry.
W20260927 23:32:29.644932 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 106: this message will repeat every 5th retry.
I20260927 23:32:29.813572 7355 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:29.814007 7355 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:29.816526 7355 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:29.818022 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:32:29.818507 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:32:29.818931 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:29.819537 7355 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:29.862520 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 101: this message will repeat every 5th retry.
I20260927 23:32:30.046514 7390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:32:30.047129 7390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:32:30.049098 7390 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:32:30.051764 4615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:32:30.054939 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:32:30.055377 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:32:30.055692 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:32:30.056370 7390 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:32:30.157044 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 106: this message will repeat every 5th retry.
I20260927 23:32:30.364625 7373 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:30.365151 7373 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:30.367197 7373 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:30.368885 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:32:30.369334 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:32:30.369625 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:30.370246 7373 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:30.613379 5015 debug-util.cc:398] Leaking SignalData structure 0x7b0800043680 after lost signal to thread 4889
W20260927 23:32:30.614706 5015 debug-util.cc:398] Leaking SignalData structure 0x7b08000af8c0 after lost signal to thread 5018
W20260927 23:32:30.679464 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 106: this message will repeat every 5th retry.
I20260927 23:32:31.158519 7401 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:31.159647 7401 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:31.162827 7401 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:31.164548 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:32:31.165665 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:32:31.166074 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:31.167032 7401 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:31.257462 7401 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:31.258044 7401 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:31.259904 7401 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:31.261639 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:32:31.262148 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:32:31.262519 5051 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:31.263165 7401 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:31.406618 7373 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:31.407363 7373 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:31.411154 7373 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:31.412609 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:32:31.415432 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:32:31.415756 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:31.416509 7373 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:31.687760 7406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:32:31.688215 7406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:32:31.696437 7406 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:32:31.722790 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:32:31.723426 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:32:31.723879 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:32:31.724560 7406 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:32:31.833681 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 106: this message will repeat every 5th retry.
W20260927 23:32:32.188937 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 106: this message will repeat every 5th retry.
W20260927 23:32:32.500535 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 106: this message will repeat every 5th retry.
W20260927 23:32:32.777704 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 106: this message will repeat every 5th retry.
I20260927 23:32:32.820885 7384 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:32.821393 7384 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:32.823104 7384 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:32.824944 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:32:32.825371 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:32:32.825666 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:32.826279 7384 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:33.535676 7355 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:33.536247 7355 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:33.538084 7355 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:33.539695 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:32:33.540154 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:32:33.540503 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:33.541114 7355 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:33.674587 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 106: this message will repeat every 5th retry.
I20260927 23:32:34.531484 7416 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:32:34.531829 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 106: this message will repeat every 5th retry.
I20260927 23:32:34.532013 7416 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:34.535930 7416 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:34.537827 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:32:34.538772 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:32:34.539270 5051 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:34.540011 7416 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:34.739041 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 106: this message will repeat every 5th retry.
I20260927 23:32:34.813050 7384 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:34.813563 7384 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:34.815469 7384 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:34.817236 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:32:34.817721 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:32:34.818123 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:34.818652 7384 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:35.043288 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 106: this message will repeat every 5th retry.
I20260927 23:32:35.263899 7441 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:32:35.264780 7441 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:32:35.267257 7441 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:32:35.271286 4615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111) [suppressed 3 similar messages]
W20260927 23:32:35.275606 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:32:35.276364 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:32:35.276726 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:32:35.277417 7441 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:32:35.328217 7436 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:35.328785 7436 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:32:35.332480 7436 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:32:35.334615 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:32:35.337016 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:32:35.337428 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:35.338510 7436 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:35.645136 7436 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:35.646421 7436 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:35.649092 7436 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:35.650610 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:32:35.651201 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:32:35.651497 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:35.652123 7436 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:35.698366 7436 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:35.698868 7436 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:35.700941 7436 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:35.703132 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:32:35.703703 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:32:35.704049 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:35.704789 7436 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:35.937898 7425 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:35.938367 7425 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:35.940166 7425 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:35.943132 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:32:35.944058 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:32:35.944451 5051 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:35.945096 7425 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:35.947911 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 106: this message will repeat every 5th retry.
W20260927 23:32:36.048061 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 106: this message will repeat every 5th retry.
I20260927 23:32:36.076946 7436 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:36.077414 7436 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:36.079257 7436 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:36.080873 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:32:36.081341 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:32:36.081606 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:36.082130 7436 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:36.219285 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 106: this message will repeat every 5th retry.
W20260927 23:32:36.398010 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 111: this message will repeat every 5th retry.
I20260927 23:32:36.550381 7451 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:36.550834 7451 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:36.553016 7451 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:36.554423 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:32:36.555073 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:32:36.555359 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:36.555894 7451 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:36.628269 7451 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:36.628926 7451 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:36.631120 7451 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:36.633191 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:32:36.634063 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:32:36.634449 5051 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:36.635098 7451 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:32:36.711611 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 106: this message will repeat every 5th retry.
W20260927 23:32:37.207973 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 111: this message will repeat every 5th retry.
I20260927 23:32:37.352708 7452 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:37.353255 7452 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:37.355316 7452 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:37.357739 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:32:37.358330 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:32:37.358700 5051 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:37.359418 7452 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:37.518991 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 111: this message will repeat every 5th retry.
W20260927 23:32:37.803825 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 111: this message will repeat every 5th retry.
I20260927 23:32:38.201277 7452 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:38.201881 7452 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:38.204020 7452 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:38.205216 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:32:38.205857 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:32:38.206169 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:38.206740 7452 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:38.234066 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 111: this message will repeat every 5th retry.
I20260927 23:32:38.333801 7457 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:38.334322 7457 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:38.336549 7457 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:38.337913 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:32:38.338586 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:32:38.338864 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:38.339466 7457 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:38.906852 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 111: this message will repeat every 5th retry.
W20260927 23:32:38.969877 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 111: this message will repeat every 5th retry.
I20260927 23:32:39.281728 7451 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:39.282243 7451 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:39.284101 7451 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:39.285755 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:32:39.286024 7451 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260927 23:32:39.286248 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:32:39.286583 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:39.286451 7451 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:39.287242 7457 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:39.288282 7451 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:39.289572 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:32:39.290002 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:32:39.290277 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:39.290773 7451 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:39.337227 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 111: this message will repeat every 5th retry.
W20260927 23:32:39.536613 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 106: this message will repeat every 5th retry.
W20260927 23:32:39.779633 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 111: this message will repeat every 5th retry.
W20260927 23:32:39.930873 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 111: this message will repeat every 5th retry.
I20260927 23:32:40.105074 7457 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:40.105548 7457 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:40.107331 7457 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:40.108882 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:32:40.109325 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:32:40.109702 5051 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:40.110265 7457 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:40.392349 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 111: this message will repeat every 5th retry.
I20260927 23:32:41.046126 7452 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:41.046986 7452 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:41.049772 7452 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:41.051614 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:32:41.052340 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:32:41.052647 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:41.053272 7452 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:32:41.508935 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 111: this message will repeat every 5th retry.
I20260927 23:32:41.943073 7484 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:41.943533 7484 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:41.945335 7484 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:41.947178 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:32:41.947574 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:32:41.947849 5051 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:41.948509 7484 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:42.044728 7452 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:42.045356 7452 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:42.047214 7452 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:42.048874 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:32:42.049325 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:32:42.049618 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:42.050186 7452 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:42.221856 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 111: this message will repeat every 5th retry.
I20260927 23:32:42.296136 7452 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:42.296830 7452 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:42.301190 7452 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:42.301972 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:32:42.303944 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:32:42.304322 5051 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:42.305186 7451 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:42.691009 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 111: this message will repeat every 5th retry.
W20260927 23:32:43.093482 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 111: this message will repeat every 5th retry.
I20260927 23:32:43.104701 7497 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:43.105230 7497 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:43.106950 7497 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:43.108618 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:32:43.108990 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:32:43.109306 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:43.109872 7497 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:43.113368 7497 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:43.113749 7497 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:43.115568 7497 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:43.117162 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:32:43.117617 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:32:43.117915 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:43.118505 7497 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:43.254079 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 111: this message will repeat every 5th retry.
I20260927 23:32:43.428411 7484 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:43.428911 7484 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:43.430786 7484 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:43.440905 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:32:43.445837 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:32:43.446317 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:43.447715 7484 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:43.761665 7508 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:43.762253 7508 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:32:43.766185 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:32:43.769562 7508 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:43.771798 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:32:43.772569 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:43.774080 7508 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:44.361524 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 111: this message will repeat every 5th retry.
W20260927 23:32:44.486995 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 111: this message will repeat every 5th retry.
W20260927 23:32:44.951851 4874 debug-util.cc:398] Leaking SignalData structure 0x7b0800105d80 after lost signal to thread 4744
W20260927 23:32:44.953002 4874 debug-util.cc:398] Leaking SignalData structure 0x7b08000f9fe0 after lost signal to thread 4877
W20260927 23:32:45.519209 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 111: this message will repeat every 5th retry.
W20260927 23:32:45.833839 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 111: this message will repeat every 5th retry.
W20260927 23:32:46.063122 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 116: this message will repeat every 5th retry.
I20260927 23:32:46.097637 7484 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:46.098097 7484 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:32:46.100565 7484 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:32:46.101809 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:32:46.102486 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:32:46.102834 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:46.103358 7484 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:46.235510 7484 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:46.236052 7484 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:46.237843 7484 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:46.239610 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:32:46.240041 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:32:46.240381 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:46.240960 7484 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:46.380545 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 111: this message will repeat every 5th retry.
I20260927 23:32:46.704151 7535 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:46.704766 7535 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:46.706866 7535 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:46.710444 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:32:46.711074 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:32:46.711506 5051 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:46.712773 7535 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:46.885193 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 111: this message will repeat every 5th retry.
W20260927 23:32:47.207366 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 111: this message will repeat every 5th retry.
W20260927 23:32:47.427675 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:47.441182 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 116: this message will repeat every 5th retry.
I20260927 23:32:47.594911 7535 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:47.595428 7535 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:47.597194 7535 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:47.598750 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:32:47.599227 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:32:47.599541 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:47.600080 7535 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:47.766090 7535 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:47.766589 7535 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:32:47.768365 7535 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:32:47.769992 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:32:47.770402 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:32:47.770671 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:47.771238 7535 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:48.025301 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 116: this message will repeat every 5th retry.
I20260927 23:32:48.169094 7508 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:48.169701 7508 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:48.171832 7508 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:48.176501 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:32:48.178262 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:32:48.178761 5051 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:48.179630 7508 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:48.531320 7484 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:48.531937 7484 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:48.534185 7484 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:48.539741 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:32:48.549504 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:32:48.549896 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:48.550506 7484 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:48.648496 7484 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:48.648970 7484 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:48.650785 7484 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:48.652618 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:32:48.653096 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:32:48.653448 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:48.654070 7484 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:48.663543 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:48.731539 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 111: this message will repeat every 5th retry.
W20260927 23:32:48.983414 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:49.048532 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 116: this message will repeat every 5th retry.
I20260927 23:32:49.222051 7554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:32:49.222666 7554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:32:49.224649 7554 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:32:49.227654 4615 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:32:49.227654 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:32:49.231226 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:32:49.231779 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:32:49.232229 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:32:49.233000 7554 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:32:49.311373 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:49.417757 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:49.646570 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 116: this message will repeat every 5th retry.
I20260927 23:32:49.753449 7484 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:49.753990 7484 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:49.755716 7484 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:49.757246 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:32:49.757714 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:32:49.757966 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:49.758441 7484 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:49.782285 7508 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:49.782755 7508 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:49.784291 7508 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:49.785914 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:32:49.786314 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:32:49.786636 5051 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:49.787233 7508 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:49.981474 7484 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:49.981902 7484 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:49.983366 7484 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:49.984901 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:32:49.985308 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:32:49.985574 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:49.986122 7484 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:50.270185 7553 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:50.270684 7553 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:50.272532 7553 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:50.274092 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:32:50.274526 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:32:50.274833 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:50.275403 7553 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:50.375547 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 116: this message will repeat every 5th retry.
I20260927 23:32:50.679463 7553 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:50.680030 7553 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:50.682176 7553 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:50.684494 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:32:50.684970 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:32:50.685326 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:50.685930 7553 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:50.858703 7553 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:50.859174 7553 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:32:50.861094 7553 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:32:50.864001 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:32:50.864997 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:32:50.865371 5054 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:50.865926 7553 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:51.588855 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:51.711525 5164 heartbeater.cc:644] Failed to heartbeat to 127.2.57.61:46527 (5251 consecutive failures): Network error: Failed to ping master at 127.2.57.61:46527: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:32:51.989918 7535 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:51.990567 7535 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:51.992594 7535 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:51.995904 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:32:51.996780 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:32:51.997488 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:51.998334 7535 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:52.090685 7508 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:52.091207 7508 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:52.093408 7508 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:52.100808 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:32:52.103091 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:32:52.103448 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:52.104096 7508 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:52.203003 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:52.404548 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:52.837181 5021 heartbeater.cc:644] Failed to heartbeat to 127.2.57.62:35155 (436 consecutive failures): Network error: Failed to ping master at 127.2.57.62:35155: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:32:53.000355 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:53.074371 4879 heartbeater.cc:644] Failed to heartbeat to 127.2.57.61:46527 (444 consecutive failures): Network error: Failed to ping master at 127.2.57.61:46527: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:32:53.234275 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 116: this message will repeat every 5th retry.
I20260927 23:32:53.335744 7568 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:53.336349 7568 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:53.338208 7568 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:53.339939 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:32:53.341037 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:32:53.341562 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:53.343739 7568 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:53.713478 7568 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:53.713986 7568 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:53.716035 7568 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:53.718113 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:32:53.718590 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:32:53.719000 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:53.719720 7568 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:54.376945 7599 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:54.377409 7599 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:54.379176 7599 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:54.380962 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:32:54.381418 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:32:54.381696 5051 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:54.382293 7599 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:54.626075 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:54.747010 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:55.460212 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 116: this message will repeat every 5th retry.
I20260927 23:32:55.558935 7568 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:55.559473 7568 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:55.561589 7568 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:55.563330 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:32:55.563793 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:32:55.564100 5051 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:55.564766 7568 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:55.756157 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 121: this message will repeat every 5th retry.
I20260927 23:32:55.888736 7606 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:55.889210 7606 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:55.890913 7606 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:55.892289 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:32:55.892791 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:32:55.893096 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:55.893705 7606 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:55.916813 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:56.008119 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 116: this message will repeat every 5th retry.
I20260927 23:32:56.671097 7568 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:56.671613 7568 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:32:56.673853 7568 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:56.676447 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:32:56.677220 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:32:56.677603 5051 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:56.678687 7568 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:56.874387 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:57.070535 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 116: this message will repeat every 5th retry.
I20260927 23:32:57.264498 7568 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:57.265036 7568 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:57.267004 7568 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:57.268610 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:32:57.269179 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:32:57.269515 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:57.270220 7568 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:32:57.652820 7621 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:57.653298 7621 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:32:57.654982 7621 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:32:57.656535 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:32:57.657011 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:32:57.657301 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:57.657886 7621 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:57.684278 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 121: this message will repeat every 5th retry.
I20260927 23:32:57.857012 7621 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:57.857447 7621 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:57.859247 7621 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:57.860635 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:32:57.861037 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:32:57.861299 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:57.861815 7621 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:57.983115 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:32:58.541510 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 121: this message will repeat every 5th retry.
I20260927 23:32:58.737262 7630 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:58.737797 7630 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:32:58.740041 7630 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:32:58.741227 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:32:58.741703 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:32:58.742045 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:58.742669 7630 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:58.778582 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 121: this message will repeat every 5th retry.
I20260927 23:32:58.827111 7630 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:32:58.827577 7630 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:32:58.829363 7630 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:32:58.831348 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:32:58.831776 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:32:58.832137 5054 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:32:58.832792 7630 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:32:58.864611 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 116: this message will repeat every 5th retry.
W20260927 23:32:59.012352 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:32:59.213992 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:32:59.376639 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:32:59.544049 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:32:59.798528 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:33:00.229159 4615 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ac88a4e4bee64586bef7b6b409a63f4c consensus state has no leader
I20260927 23:33:00.385102 7620 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:00.385651 7620 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:00.387347 7620 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:00.389134 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:33:00.390100 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:33:00.390486 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:00.392333 7620 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:00.500097 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 121: this message will repeat every 5th retry.
I20260927 23:33:00.617435 7630 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:00.617868 7630 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:00.619563 7630 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:00.621168 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:33:00.621644 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:33:00.621933 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:00.622661 7630 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:00.840406 7650 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:00.840931 7650 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:33:00.846271 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:33:00.847276 7650 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:00.848768 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:33:00.849186 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:00.849843 7650 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:33:01.006487 7630 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:01.006963 7630 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:01.008884 7630 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:01.010180 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:33:01.010763 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:33:01.011137 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:01.011849 7630 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:01.251847 7650 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:01.252701 7650 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:01.255373 7650 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:01.257463 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:33:01.258116 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:33:01.258395 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:01.259135 7650 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:01.433313 7659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:33:01.434115 7659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:33:01.436066 7659 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:33:01.441038 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:33:01.444716 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:33:01.445206 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:33:01.445528 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:33:01.446149 7659 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:33:01.527208 7650 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:01.527866 7650 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:01.529834 7650 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:01.531556 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:33:01.532152 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:33:01.532593 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:01.533358 7650 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:01.811920 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:33:02.030540 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:33:02.167902 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 121: this message will repeat every 5th retry.
I20260927 23:33:02.310941 7665 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:02.311463 7665 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:02.313979 7665 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:02.315237 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:33:02.315779 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:33:02.316181 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:02.316939 7650 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:02.654917 7621 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:02.655404 7621 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:02.657423 7621 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:02.658805 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:33:02.659489 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:33:02.659791 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:02.660394 7621 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:02.674672 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:33:03.215083 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 121: this message will repeat every 5th retry.
I20260927 23:33:03.556069 7665 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:03.556634 7665 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:03.558602 7665 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:03.560276 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:33:03.560771 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:33:03.561123 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:03.561841 7665 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:04.323180 7650 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:04.323644 7650 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:33:04.325354 7650 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:33:04.326748 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:33:04.327131 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:33:04.327426 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:04.328032 7650 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:04.450896 7665 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:04.451512 7665 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:04.453368 7665 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:04.454955 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:33:04.455760 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:33:04.456053 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:04.456665 7665 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:04.781950 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:33:04.858323 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:33:05.084604 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:33:05.576028 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:05.973500 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:33:06.075369 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:33:06.288367 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 327adcc2a27f42acb3a77d49fbf73796 (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 327adcc2a27f42acb3a77d49fbf73796 consensus state has no leader
W20260927 23:33:06.996549 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 121: this message will repeat every 5th retry.
I20260927 23:33:07.339322 7688 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:07.339792 7688 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:07.341922 7688 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:07.343693 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:33:07.344246 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:33:07.344532 5051 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:07.345042 7688 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:07.508890 7688 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:07.509418 7688 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:07.511942 7688 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:07.513386 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:33:07.513832 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:33:07.514196 5051 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:07.514853 7688 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:07.565871 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 121: this message will repeat every 5th retry.
I20260927 23:33:07.615998 7688 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:07.616575 7688 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:07.618417 7688 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:07.619782 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:33:07.620222 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:33:07.620496 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:07.621001 7688 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:07.644901 7621 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:07.645367 7621 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:07.647104 7621 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:07.648478 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:33:07.650239 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:33:07.650876 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:07.651459 7621 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:07.953621 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet 1766d227273e4c7592c4f0a20f2f7b17 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet 1766d227273e4c7592c4f0a20f2f7b17 consensus state has no leader
I20260927 23:33:08.290690 7713 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:08.291191 7713 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:08.293017 7713 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:08.294559 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:33:08.294935 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:33:08.295245 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:08.295847 7713 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:08.761628 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 121: this message will repeat every 5th retry.
W20260927 23:33:08.786664 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 126: this message will repeat every 5th retry.
I20260927 23:33:08.934710 7720 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:33:08.935284 7720 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:33:08.937077 7720 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:33:08.939656 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:33:08.943109 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:33:08.943651 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:33:08.943933 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:33:08.944567 7720 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:33:08.997545 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:09.029439 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:09.050717 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:09.169581 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:09.179224 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:09.275023 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 126: this message will repeat every 5th retry.
I20260927 23:33:09.529570 7713 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:09.530179 7713 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:09.532258 7713 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:09.534034 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:33:09.534482 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:33:09.534884 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:09.535694 7713 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:09.702895 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:10.175164 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:10.512902 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:10.909727 4617 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet b9f07e8a37404636bda765146b20fc60 (table default.table_23bd70063a25487ab35a76f05dd4b81b [id=ec636e7b20704b56bd3191822c076863], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet b9f07e8a37404636bda765146b20fc60 consensus state has no leader
I20260927 23:33:11.062403 7740 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:11.062951 7740 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:11.065146 7740 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:11.066618 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:33:11.067068 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:33:11.067384 5054 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:11.067924 7740 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:11.693775 7713 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:11.694419 7713 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:11.696616 7713 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:11.698333 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:33:11.698909 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:33:11.699199 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:11.699811 7713 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:11.723626 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 126: this message will repeat every 5th retry.
I20260927 23:33:11.978514 7750 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:11.979036 7750 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:11.981400 7750 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:11.983069 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:33:11.983525 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:33:11.983811 5051 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:11.984414 7750 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:12.327898 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:12.331700 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 126: this message will repeat every 5th retry.
I20260927 23:33:12.550896 7740 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:12.551443 7740 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:12.553596 7740 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:12.555321 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:33:12.555804 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:33:12.556159 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:12.556810 7740 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:12.598280 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 126: this message will repeat every 5th retry.
I20260927 23:33:12.727710 7713 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:12.728225 7713 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:12.730173 7713 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:12.731637 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:33:12.732120 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:33:12.732508 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:12.733155 7713 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:13.097859 7754 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:13.098387 7754 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:33:13.102684 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:33:13.102795 7754 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:13.103336 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:33:13.103715 5051 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:13.104413 7754 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:13.111657 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:14.800065 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:14.963089 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:15.078011 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 131: this message will repeat every 5th retry.
I20260927 23:33:15.143414 7754 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:15.143976 7754 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:33:15.145810 7754 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:33:15.147658 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:33:15.148080 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:33:15.148430 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:15.149065 7754 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:15.376417 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:15.795575 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:16.137595 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:16.239564 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 126: this message will repeat every 5th retry.
I20260927 23:33:16.732030 7775 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:16.732594 7775 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:16.735339 7775 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:16.736452 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:33:16.739089 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:33:16.739451 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:16.740485 7775 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:17.087867 7779 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:17.088459 7779 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:17.090580 7779 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:17.092168 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:33:17.092638 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:33:17.092926 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:17.093513 7779 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:17.130481 7780 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:17.130959 7780 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:17.133100 7780 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:17.135226 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:33:17.137173 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:33:17.137493 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:17.138078 7780 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:18.024639 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:18.192147 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 131: this message will repeat every 5th retry.
I20260927 23:33:18.466939 7786 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:18.467514 7786 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:18.469640 7786 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:18.471306 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:33:18.471982 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:33:18.472257 5051 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:18.472887 7786 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:18.737913 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:18.743541 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 126: this message will repeat every 5th retry.
W20260927 23:33:18.761032 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:19.014577 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:19.029467 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 131: this message will repeat every 5th retry.
I20260927 23:33:19.112603 7797 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:19.113149 7797 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:19.114790 7797 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:19.116163 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:33:19.116772 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:33:19.117071 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:19.117596 7797 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:19.291972 7797 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:19.292490 7797 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:19.294176 7797 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:19.295635 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:33:19.296020 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:33:19.296340 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:19.296885 7797 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:19.351203 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:19.426055 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:19.654086 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 131: this message will repeat every 5th retry.
I20260927 23:33:19.799765 7792 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:19.800220 7792 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:33:19.802009 7792 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:33:19.803586 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:33:19.804015 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:33:19.804349 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:19.804939 7792 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:19.882524 7786 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:19.883231 7786 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:19.885128 7786 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:19.888427 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:33:19.889708 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:33:19.890100 5051 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:19.890816 7805 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:33:19.983793 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 131: this message will repeat every 5th retry.
I20260927 23:33:20.434461 7786 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:20.435006 7786 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:20.436733 7786 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:20.438146 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:33:20.438616 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:33:20.438913 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:20.439524 7786 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:20.511664 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 131: this message will repeat every 5th retry.
I20260927 23:33:20.862684 7810 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:20.863142 7810 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:20.865080 7810 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:20.866458 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:33:20.866909 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:33:20.867197 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:20.867795 7810 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:21.426610 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:21.869045 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:22.257664 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:22.293423 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:22.454265 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 131: this message will repeat every 5th retry.
I20260927 23:33:22.498339 7810 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:22.499071 7810 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:22.501328 7810 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:22.503260 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:33:22.503918 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:33:22.504310 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:22.505112 7810 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:22.786482 7828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:33:22.787076 7828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:33:22.788959 7828 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:33:22.791668 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:33:22.795077 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:33:22.795590 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:33:22.795881 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:33:22.796475 7828 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:33:23.641917 7834 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:23.642514 7834 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:23.645414 7834 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:23.647202 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:33:23.647665 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:33:23.648029 5051 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:23.649127 7834 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:24.768008 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:24.889297 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:25.011211 5015 debug-util.cc:398] Leaking SignalData structure 0x7b08001225e0 after lost signal to thread 4889
W20260927 23:33:25.012755 5015 debug-util.cc:398] Leaking SignalData structure 0x7b0800121ee0 after lost signal to thread 5018
I20260927 23:33:25.019177 7810 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:25.019682 7810 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:25.021795 7810 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:25.023663 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:33:25.024204 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:33:25.024549 5051 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:25.025147 7810 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:25.183372 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 136: this message will repeat every 5th retry.
I20260927 23:33:25.215622 7810 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:25.216077 7810 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:25.217717 7810 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:25.219277 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:33:25.219682 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:33:25.220065 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:25.220988 7810 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:25.261523 7847 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:25.262072 7847 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:25.263765 7847 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:25.265520 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:33:25.269099 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:33:25.269443 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:25.270066 7847 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:25.498021 7846 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:25.498582 7846 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:25.500815 7846 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:25.502533 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:33:25.503073 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:33:25.503378 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:25.504016 7846 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:25.908131 7846 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:25.908623 7846 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:25.910337 7846 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:25.911742 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:33:25.912163 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:33:25.912467 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:25.913017 7846 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:26.012586 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:26.019847 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:26.047166 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 131: this message will repeat every 5th retry.
I20260927 23:33:26.401834 7846 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:26.402361 7846 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:26.404153 7846 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:26.405958 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:33:26.406509 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:33:26.406868 5051 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:26.407610 7846 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:26.493415 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:26.693625 5159 debug-util.cc:398] Leaking SignalData structure 0x7b08000dde80 after lost signal to thread 5032
W20260927 23:33:26.695020 5159 debug-util.cc:398] Leaking SignalData structure 0x7b080013bea0 after lost signal to thread 5162
I20260927 23:33:26.834370 7847 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:26.834875 7847 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:33:26.837491 7847 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:33:26.838425 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:33:26.839025 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:33:26.839366 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:26.840047 7847 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:26.975651 7847 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:26.976159 7847 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:26.977932 7847 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:26.979653 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:33:26.980079 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:33:26.980397 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:26.980989 7847 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:27.895979 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:28.166972 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 131: this message will repeat every 5th retry.
I20260927 23:33:28.504854 7847 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:28.505360 7847 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:28.507184 7847 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:28.508803 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:33:28.509390 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:33:28.509708 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:28.510331 7847 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:28.713059 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:28.859113 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 131: this message will repeat every 5th retry.
W20260927 23:33:29.027258 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 136: this message will repeat every 5th retry.
I20260927 23:33:29.220062 7859 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:29.220558 7859 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:29.222322 7859 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:29.223603 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:33:29.224222 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:33:29.224607 5054 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:29.225194 7859 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:29.277896 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:29.289757 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:29.314111 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:29.390869 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:29.410637 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 136: this message will repeat every 5th retry.
I20260927 23:33:29.814694 7870 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:29.815209 7870 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:29.817368 7870 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:29.818722 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:33:29.819120 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:33:29.819417 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:29.819974 7870 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:29.997041 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 136: this message will repeat every 5th retry.
I20260927 23:33:30.009991 7870 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:30.010407 7870 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:30.011972 7870 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:30.013526 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:33:30.013962 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:33:30.014262 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:30.014794 7870 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:30.217881 7847 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:30.218485 7847 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:30.220482 7847 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:30.222205 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:33:30.222721 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:33:30.223111 5051 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:30.223755 7847 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:30.498096 7876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:33:30.498831 7876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:33:30.500793 7876 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:33:30.503575 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:33:30.510191 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:33:30.510704 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:33:30.511107 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:33:30.511816 7876 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:33:30.889539 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 136: this message will repeat every 5th retry.
I20260927 23:33:30.998446 7870 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:31.000634 7870 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:31.002849 7870 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:31.004406 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:33:31.004869 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:33:31.005244 5051 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:31.005826 7870 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:31.336966 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 136: this message will repeat every 5th retry.
I20260927 23:33:31.866804 7886 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:31.867432 7886 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:31.870097 7886 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:31.871685 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:33:31.872252 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:33:31.872617 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:31.873296 7886 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:32.025175 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:32.284845 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:32.575760 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 136: this message will repeat every 5th retry.
I20260927 23:33:33.022717 7886 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:33.023309 7886 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:33.025434 7886 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:33.027238 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:33:33.027767 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:33:33.028096 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:33.028694 7886 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:33.098845 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 136: this message will repeat every 5th retry.
I20260927 23:33:34.422541 7886 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:34.423033 7886 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:34.424839 7886 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:34.426589 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:33:34.427065 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:33:34.427386 5051 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:34.427989 7886 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:34.497951 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:34.637159 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:34.797736 5015 debug-util.cc:398] Leaking SignalData structure 0x7b0800082a60 after lost signal to thread 4889
W20260927 23:33:34.799232 5015 debug-util.cc:398] Leaking SignalData structure 0x7b080011c560 after lost signal to thread 5018
I20260927 23:33:35.036731 7887 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:35.037343 7887 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:35.040386 7887 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:35.041363 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:33:35.045488 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:33:35.045868 5051 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:35.046845 7887 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:35.349509 7887 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:35.349987 7887 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:33:35.351651 7887 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:33:35.355878 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:33:35.356431 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:33:35.356777 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
W20260927 23:33:35.357587 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 136: this message will repeat every 5th retry.
I20260927 23:33:35.357580 7923 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:35.409076 7923 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:35.409737 7923 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:35.411852 7923 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:35.413292 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:33:35.413750 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:33:35.414052 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:35.414595 7923 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:35.564936 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:35.621446 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 136: this message will repeat every 5th retry.
I20260927 23:33:36.218645 7887 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:36.219158 7887 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:36.221130 7887 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:36.223176 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:33:36.223636 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:33:36.223946 5051 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:36.224597 7887 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:33:36.337682 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:36.492226 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 136: this message will repeat every 5th retry.
I20260927 23:33:36.706561 7886 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:36.707067 7886 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:36.709121 7886 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:36.710533 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:33:36.712137 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:33:36.712533 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:36.713165 7886 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:37.360505 7936 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:37.361210 7936 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:37.363461 7936 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:37.365298 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:33:37.369033 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:33:37.371546 5051 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:37.373898 7938 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:38.094782 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:38.216882 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:38.416477 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:38.646234 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:38.992733 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:39.151530 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 136: this message will repeat every 5th retry.
W20260927 23:33:39.426335 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:39.448269 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:39.714510 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 141: this message will repeat every 5th retry.
I20260927 23:33:39.762992 7887 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:39.763569 7887 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:39.765929 7887 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:39.767155 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:33:39.767967 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:33:39.768359 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:39.769085 7887 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:39.853444 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:39.952494 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 141: this message will repeat every 5th retry.
I20260927 23:33:40.427733 7887 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:40.428330 7887 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:40.430326 7887 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:40.432026 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:33:40.432544 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:33:40.432919 5051 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:40.434002 7887 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:40.823917 7887 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:40.824405 7887 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:40.825984 7887 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:40.827792 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:33:40.829191 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:33:40.829581 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:40.830231 7887 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:41.027124 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:41.786103 4661 debug-util.cc:398] Leaking SignalData structure 0x7b080005bb00 after lost signal to thread 4593
W20260927 23:33:41.787015 4661 debug-util.cc:398] Leaking SignalData structure 0x7b080009b660 after lost signal to thread 4664
I20260927 23:33:41.800544 7954 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:41.801532 7954 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:41.805241 7954 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:41.806525 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:33:41.807257 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:33:41.807596 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:41.808346 7954 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:41.904778 7954 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:41.905220 7954 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:41.906905 7954 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:41.908790 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:33:41.909310 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:33:41.909631 5051 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:41.910231 7954 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:42.076005 7954 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:42.076624 7954 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:42.078889 7954 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:42.080600 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:33:42.082203 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:33:42.082576 5054 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:42.083281 7954 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:42.204178 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:42.358086 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:42.625288 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 141: this message will repeat every 5th retry.
I20260927 23:33:42.664425 7938 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:42.664968 7938 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:42.666872 7938 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:42.668283 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:33:42.668707 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:33:42.669025 5051 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:42.669718 7938 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:42.683005 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 141: this message will repeat every 5th retry.
I20260927 23:33:43.067814 7887 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:43.068344 7887 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
W20260927 23:33:43.072805 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:33:43.071889 7887 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:43.075170 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:33:43.075491 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:43.076015 7887 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:43.163727 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 141: this message will repeat every 5th retry.
I20260927 23:33:43.376657 7938 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:43.377149 7938 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:43.379829 7938 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:43.380540 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:33:43.380941 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:33:43.381320 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:43.381987 7938 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:43.649914 7954 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:43.650418 7954 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
W20260927 23:33:43.655031 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:33:43.657738 7954 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:43.660256 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:33:43.660713 5054 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:43.662106 7970 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:43.679328 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 146: this message will repeat every 5th retry.
I20260927 23:33:43.849450 7954 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:43.850224 7954 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:43.853963 7954 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:43.856033 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:33:43.856525 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:33:43.856825 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:43.857362 7954 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:43.879890 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 141: this message will repeat every 5th retry.
I20260927 23:33:43.975498 7954 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:43.975937 7954 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:33:43.978086 7954 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:33:43.981289 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:33:43.981814 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:33:43.982174 5054 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:43.983083 7970 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:45.166533 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:45.553398 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 141: this message will repeat every 5th retry.
I20260927 23:33:45.560477 7954 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:45.560954 7954 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:45.562564 7954 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:45.563959 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:33:45.564357 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:33:45.564741 5054 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:45.565344 7954 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:45.657807 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:45.979879 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:45.992666 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 141: this message will repeat every 5th retry.
I20260927 23:33:46.615486 7984 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:46.615989 7984 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:46.618422 7984 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:46.619715 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:33:46.620325 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:33:46.620618 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:46.621160 7984 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:47.070385 7970 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:47.070927 7970 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:47.075150 7970 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:47.076830 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:33:47.077598 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:33:47.078123 5054 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:47.078855 7970 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 23:33:47.244989 7970 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:47.245534 7970 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:47.247216 7970 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:47.248783 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:33:47.249167 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:33:47.249558 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:47.250273 7970 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:47.587227 7984 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:47.587743 7984 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:47.589465 7984 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:47.590766 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:33:47.591261 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:33:47.592337 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:47.593366 7984 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:47.712604 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:47.780037 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:48.130167 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:48.386157 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:48.440512 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 141: this message will repeat every 5th retry.
W20260927 23:33:48.550473 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 146: this message will repeat every 5th retry.
I20260927 23:33:49.150272 7995 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:49.150852 7995 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:49.153043 7995 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:49.154541 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:33:49.155478 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:33:49.155828 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:49.156548 7995 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:49.340018 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 146: this message will repeat every 5th retry.
I20260927 23:33:49.519966 8002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:33:49.520632 8002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:33:49.526230 8002 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:33:49.526165 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:33:49.529727 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:33:49.530260 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
I20260927 23:33:49.530623 4616 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:33:49.531239 8002 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260927 23:33:49.618660 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:49.676263 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 146: this message will repeat every 5th retry.
I20260927 23:33:49.924358 7995 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:49.924854 7995 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:49.926877 7995 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:49.928139 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:33:49.928812 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:33:49.929095 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:49.929936 7995 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:50.058419 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:50.111397 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 146: this message will repeat every 5th retry.
I20260927 23:33:50.689110 8000 raft_consensus.cc:482] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:50.689631 8000 raft_consensus.cc:504] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:50.691273 8000 leader_election.cc:288] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:50.693084 5051 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
W20260927 23:33:50.693498 5054 leader_election.cc:341] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: f171cc1f0094480a9cc6357eab93dff1
I20260927 23:33:50.693866 5054 leader_election.cc:302] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:50.694669 8008 raft_consensus.cc:2726] T f171cc1f0094480a9cc6357eab93dff1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:51.454493 8008 raft_consensus.cc:482] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:51.455312 8008 raft_consensus.cc:504] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:51.457892 8008 leader_election.cc:288] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:51.460553 5054 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
W20260927 23:33:51.461187 5051 leader_election.cc:341] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: ffd4fd7dc4b14265b0eafe894b748131
I20260927 23:33:51.461606 5051 leader_election.cc:302] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:51.462502 8008 raft_consensus.cc:2726] T ffd4fd7dc4b14265b0eafe894b748131 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:51.529381 8008 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:51.529932 8008 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:51.531801 8008 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:51.533228 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:33:51.533932 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:33:51.534287 5054 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:51.534929 8008 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:51.717585 5164 heartbeater.cc:644] Failed to heartbeat to 127.2.57.61:46527 (6684 consecutive failures): Network error: Failed to ping master at 127.2.57.61:46527: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
W20260927 23:33:51.733995 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:51.798878 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 146: this message will repeat every 5th retry.
I20260927 23:33:52.262130 8000 raft_consensus.cc:482] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:52.262804 8000 raft_consensus.cc:504] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:52.266238 8000 leader_election.cc:288] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:52.268024 5054 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
W20260927 23:33:52.334916 5051 leader_election.cc:341] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 14b3852ae71c48fc830ec6939a53c18a
I20260927 23:33:52.335381 5051 leader_election.cc:302] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:52.337065 8000 raft_consensus.cc:2726] T 14b3852ae71c48fc830ec6939a53c18a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:52.521889 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:52.566615 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:52.583612 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:52.747382 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 151: this message will repeat every 5th retry.
W20260927 23:33:52.890672 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:53.263967 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:53.284837 5021 heartbeater.cc:644] Failed to heartbeat to 127.2.57.62:35155 (556 consecutive failures): Network error: Failed to ping master at 127.2.57.62:35155: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:33:53.307698 4874 debug-util.cc:398] Leaking SignalData structure 0x7b08000fe540 after lost signal to thread 4744
W20260927 23:33:53.309182 4874 debug-util.cc:398] Leaking SignalData structure 0x7b0800107140 after lost signal to thread 4877
W20260927 23:33:53.374509 4879 heartbeater.cc:644] Failed to heartbeat to 127.2.57.61:46527 (564 consecutive failures): Network error: Failed to ping master at 127.2.57.61:46527: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:33:54.488010 8008 raft_consensus.cc:482] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:54.488700 8008 raft_consensus.cc:504] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:54.490504 8008 leader_election.cc:288] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:54.492218 5054 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
W20260927 23:33:54.492666 5051 leader_election.cc:341] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c5552715138046279f00dcf497d05a72
I20260927 23:33:54.492945 5051 leader_election.cc:302] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:54.493551 8008 raft_consensus.cc:2726] T c5552715138046279f00dcf497d05a72 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:54.898402 5051 consensus_peers.cc:487] T c4deca666d4e4d25aae3ed6f13e97c4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c4deca666d4e4d25aae3ed6f13e97c4d. This is attempt 146: this message will repeat every 5th retry.
I20260927 23:33:54.912992 8008 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:54.913554 8008 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:54.915256 8008 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:54.916993 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:33:54.917661 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:33:54.918033 5051 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:54.918715 8008 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:55.198412 8000 raft_consensus.cc:482] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:55.198892 8000 raft_consensus.cc:504] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:55.200603 8000 leader_election.cc:288] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:55.202196 5051 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
W20260927 23:33:55.202760 5054 leader_election.cc:341] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 327adcc2a27f42acb3a77d49fbf73796
I20260927 23:33:55.203114 5054 leader_election.cc:302] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:55.204259 8008 raft_consensus.cc:2726] T 327adcc2a27f42acb3a77d49fbf73796 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:55.263963 8000 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:55.264485 8000 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:55.266112 8000 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:55.267655 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:33:55.268067 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:33:55.268474 5054 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:55.269035 8000 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:55.290032 8000 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:55.290513 8000 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:33:55.292141 8000 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:33:55.293713 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:33:55.294633 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:33:55.295011 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:55.295627 8000 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:56.125770 5051 consensus_peers.cc:487] T 0c8328db65eb43ebb79cd9d7a1ddb0d8 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c8328db65eb43ebb79cd9d7a1ddb0d8. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:56.314214 5051 consensus_peers.cc:487] T 2826716e5b184cc2b60d556ca2b89f4d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2826716e5b184cc2b60d556ca2b89f4d. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:56.319984 5051 consensus_peers.cc:487] T 6846550519ce44fdafbc895ddca98d8b P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6846550519ce44fdafbc895ddca98d8b. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:56.334169 5051 consensus_peers.cc:487] T 64cfbad1561a41048009b5fc819cc5de P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 64cfbad1561a41048009b5fc819cc5de. This is attempt 146: this message will repeat every 5th retry.
I20260927 23:33:56.764876 8000 raft_consensus.cc:482] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:56.765377 8000 raft_consensus.cc:504] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:56.767411 8000 leader_election.cc:288] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:56.769253 5054 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
W20260927 23:33:56.769815 5051 leader_election.cc:341] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: baae829e8b9c4e7f8c2d3a1414666fcb
I20260927 23:33:56.770284 5051 leader_election.cc:302] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:56.770946 8000 raft_consensus.cc:2726] T baae829e8b9c4e7f8c2d3a1414666fcb P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:57.069579 8008 raft_consensus.cc:482] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:57.070152 8008 raft_consensus.cc:504] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:33:57.072218 8008 leader_election.cc:288] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:57.073876 5051 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
W20260927 23:33:57.074374 5054 leader_election.cc:341] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 26c0a9168b8b44fb9283c97a4206b79b
I20260927 23:33:57.074802 5054 leader_election.cc:302] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:57.075521 8008 raft_consensus.cc:2726] T 26c0a9168b8b44fb9283c97a4206b79b P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:57.416162 5051 consensus_peers.cc:487] T 0c7f48abe7db4a5aa39892af9a3e3713 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c7f48abe7db4a5aa39892af9a3e3713. This is attempt 151: this message will repeat every 5th retry.
I20260927 23:33:57.640054 8000 raft_consensus.cc:482] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:57.640576 8000 raft_consensus.cc:504] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:57.642009 8000 leader_election.cc:288] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:57.643301 5051 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
W20260927 23:33:57.643723 5054 leader_election.cc:341] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 309465f419c24549bf5416a4503b9299
I20260927 23:33:57.643985 5054 leader_election.cc:302] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:57.644507 8000 raft_consensus.cc:2726] T 309465f419c24549bf5416a4503b9299 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:58.034075 5051 consensus_peers.cc:487] T 0a3148da66024c4684d5eaf3d60c8c58 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0a3148da66024c4684d5eaf3d60c8c58. This is attempt 151: this message will repeat every 5th retry.
W20260927 23:33:58.383869 5051 consensus_peers.cc:487] T 5073ab2faa0042569b11193952cce2ac P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5073ab2faa0042569b11193952cce2ac. This is attempt 146: this message will repeat every 5th retry.
W20260927 23:33:58.784991 5051 consensus_peers.cc:487] T 3a0c6ad197164aa98d73d357de26b488 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3a0c6ad197164aa98d73d357de26b488. This is attempt 151: this message will repeat every 5th retry.
W20260927 23:33:58.828222 5051 consensus_peers.cc:487] T 77734e994d5d40dfb3efca344d396956 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 77734e994d5d40dfb3efca344d396956. This is attempt 146: this message will repeat every 5th retry.
I20260927 23:33:59.002344 8060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d9ed71d97661411891f2d2de4cdc2269)
I20260927 23:33:59.003191 8060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d9ed71d97661411891f2d2de4cdc2269" member_type: VOTER last_known_addr { host: "127.2.57.62" port: 35155 } } peers { permanent_uuid: "f8460f73b40c41a28ec020fc04203564" member_type: VOTER last_known_addr { host: "127.2.57.61" port: 46527 } } peers { permanent_uuid: "83cd22a89b7846999c0e4dfab80ebe9d" member_type: VOTER last_known_addr { host: "127.2.57.60" port: 33749 } }
I20260927 23:33:59.006081 8060 leader_election.cc:288] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155), f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527)
W20260927 23:33:59.008580 4616 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260927 23:33:59.013556 4616 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d9ed71d97661411891f2d2de4cdc2269 (127.2.57.62:35155): Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
W20260927 23:33:59.017663 4615 leader_election.cc:334] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f8460f73b40c41a28ec020fc04203564 (127.2.57.61:46527): Network error: Client connection negotiation failed: client connection to 127.2.57.61:46527: connect: Connection refused (error 111)
I20260927 23:33:59.021083 4615 leader_election.cc:302] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83cd22a89b7846999c0e4dfab80ebe9d; no voters: d9ed71d97661411891f2d2de4cdc2269, f8460f73b40c41a28ec020fc04203564
I20260927 23:33:59.021844 8060 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83cd22a89b7846999c0e4dfab80ebe9d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260927 23:33:59.074922 8048 raft_consensus.cc:482] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:59.075469 8048 raft_consensus.cc:504] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:59.077620 8048 leader_election.cc:288] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:59.079517 5051 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
W20260927 23:33:59.079927 5054 leader_election.cc:341] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 1766d227273e4c7592c4f0a20f2f7b17
I20260927 23:33:59.080279 5054 leader_election.cc:302] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:59.080986 8048 raft_consensus.cc:2726] T 1766d227273e4c7592c4f0a20f2f7b17 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:33:59.151839 5051 consensus_peers.cc:487] T 360070544fca4f62b0aac16d5889e1bc P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 360070544fca4f62b0aac16d5889e1bc. This is attempt 151: this message will repeat every 5th retry.
W20260927 23:33:59.641081 5051 consensus_peers.cc:487] T a483b5e32d55406bb3342af93f755028 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a483b5e32d55406bb3342af93f755028. This is attempt 151: this message will repeat every 5th retry.
W20260927 23:33:59.709919 5051 consensus_peers.cc:487] T 667f1147ecae4874854935982dd19e96 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 667f1147ecae4874854935982dd19e96. This is attempt 151: this message will repeat every 5th retry.
W20260927 23:33:59.778723 5051 consensus_peers.cc:487] T 43f8e7644db24be5a758b437a5f471c0 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43f8e7644db24be5a758b437a5f471c0. This is attempt 151: this message will repeat every 5th retry.
W20260927 23:33:59.936038 5051 consensus_peers.cc:487] T e7a73bf1fc35439ea3e39ffd843ebfb1 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7a73bf1fc35439ea3e39ffd843ebfb1. This is attempt 151: this message will repeat every 5th retry.
I20260927 23:33:59.960268 8048 raft_consensus.cc:482] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:59.960839 8048 raft_consensus.cc:504] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:59.962785 8048 leader_election.cc:288] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:59.964329 5051 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
W20260927 23:33:59.964689 5054 leader_election.cc:341] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a007bdab1e3c4e40a259b97f1e43bd51
I20260927 23:33:59.964995 5054 leader_election.cc:302] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:59.965560 8048 raft_consensus.cc:2726] T a007bdab1e3c4e40a259b97f1e43bd51 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:33:59.968097 8048 raft_consensus.cc:482] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:33:59.968616 8048 raft_consensus.cc:504] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:33:59.970170 8048 leader_election.cc:288] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:33:59.971599 5051 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
W20260927 23:33:59.972029 5054 leader_election.cc:341] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b4ec12a496904c3599bdc001476ebee9
I20260927 23:33:59.972411 5054 leader_election.cc:302] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:33:59.973014 8048 raft_consensus.cc:2726] T b4ec12a496904c3599bdc001476ebee9 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:34:00.247937 4615 catalog_manager.cc:4639] Async tablet task AlterTable RPC for tablet ac88a4e4bee64586bef7b6b409a63f4c (table default.table_e141ccac1ae746388643f3ee5f04cdc1 [id=c2a82848ea2847feb1c711f38247865d], current schema version=1) failed: Not found: failed to reset TS proxy: No leader found: Tablet ac88a4e4bee64586bef7b6b409a63f4c consensus state has no leader
I20260927 23:34:00.417146 8048 raft_consensus.cc:482] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:34:00.417639 8048 raft_consensus.cc:504] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:34:00.419433 8048 leader_election.cc:288] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:34:00.420967 5051 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
W20260927 23:34:00.421464 5054 leader_election.cc:341] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: c739222645654eb39285a2ad1b6c35f5
I20260927 23:34:00.421834 5054 leader_election.cc:302] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:34:00.422461 8048 raft_consensus.cc:2726] T c739222645654eb39285a2ad1b6c35f5 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:34:00.594020 8048 raft_consensus.cc:482] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:34:00.594506 8048 raft_consensus.cc:504] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:34:00.596201 8048 leader_election.cc:288] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:34:00.598392 5054 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
W20260927 23:34:00.599084 5051 leader_election.cc:341] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: a1ae6ef92e96465fa5a414ada1b4ff51
I20260927 23:34:00.599489 5051 leader_election.cc:302] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:34:00.600392 8000 raft_consensus.cc:2726] T a1ae6ef92e96465fa5a414ada1b4ff51 P f5ec98b2972d44838fa273eb7a10d01a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260927 23:34:00.637533 5051 consensus_peers.cc:487] T 2dcb78350f4c4b73beada474bcb01169 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2dcb78350f4c4b73beada474bcb01169. This is attempt 151: this message will repeat every 5th retry.
I20260927 23:34:00.799937 8008 raft_consensus.cc:482] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:34:00.800446 8008 raft_consensus.cc:504] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:34:00.802162 8008 leader_election.cc:288] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:34:00.803915 5051 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
W20260927 23:34:00.804345 5054 leader_election.cc:341] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e56378a5d42e47d395c881a0898da495
I20260927 23:34:00.804664 5054 leader_election.cc:302] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:34:00.805225 8008 raft_consensus.cc:2726] T e56378a5d42e47d395c881a0898da495 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:34:01.355250 5051 consensus_peers.cc:487] T ac88a4e4bee64586bef7b6b409a63f4c P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac88a4e4bee64586bef7b6b409a63f4c. This is attempt 151: this message will repeat every 5th retry.
I20260927 23:34:01.597038 8000 raft_consensus.cc:482] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:34:01.597980 8000 raft_consensus.cc:504] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:34:01.600500 8000 leader_election.cc:288] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:34:01.604326 5054 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
W20260927 23:34:01.605865 5051 leader_election.cc:341] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9912e4dd70ba482398fbd831a8242f12
I20260927 23:34:01.606408 5051 leader_election.cc:302] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:34:01.607811 8069 raft_consensus.cc:2726] T 9912e4dd70ba482398fbd831a8242f12 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:34:02.278370 5051 consensus_peers.cc:487] T 38cd408c3dc34fabbab5906286e7580a P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 38cd408c3dc34fabbab5906286e7580a. This is attempt 151: this message will repeat every 5th retry.
I20260927 23:34:02.365860 8000 raft_consensus.cc:482] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:34:02.366304 8000 raft_consensus.cc:504] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } }
I20260927 23:34:02.368613 8000 leader_election.cc:288] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717), 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571)
W20260927 23:34:02.370013 5054 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
W20260927 23:34:02.370527 5051 leader_election.cc:341] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e0f9998e6eb24f0a8fa217985ce3b07d
I20260927 23:34:02.370819 5051 leader_election.cc:302] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:34:02.371326 8000 raft_consensus.cc:2726] T e0f9998e6eb24f0a8fa217985ce3b07d P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:34:02.417507 5051 consensus_peers.cc:487] T df6ea36da9f6455bb399c2229dcb3790 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: df6ea36da9f6455bb399c2229dcb3790. This is attempt 151: this message will repeat every 5th retry.
W20260927 23:34:02.459026 5051 consensus_peers.cc:487] T 7a2477e4ce30497fa94f2dfdb3c31a3d P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7a2477e4ce30497fa94f2dfdb3c31a3d. This is attempt 151: this message will repeat every 5th retry.
W20260927 23:34:02.501745 5051 consensus_peers.cc:487] T 6c3c55f368dd4ba991daded55d2e3069 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6c3c55f368dd4ba991daded55d2e3069. This is attempt 151: this message will repeat every 5th retry.
I20260927 23:34:02.515112 8000 raft_consensus.cc:482] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:34:02.515535 8000 raft_consensus.cc:504] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:34:02.517184 8000 leader_election.cc:288] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:34:02.518676 5051 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
W20260927 23:34:02.519157 5054 leader_election.cc:341] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: b373bde60f734d049c4a45255e2b2e2a
I20260927 23:34:02.519556 5054 leader_election.cc:302] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:34:02.520135 8000 raft_consensus.cc:2726] T b373bde60f734d049c4a45255e2b2e2a P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260927 23:34:02.550536 5051 consensus_peers.cc:487] T cc208e8764244ac68b2a73be1b3d957e P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cc208e8764244ac68b2a73be1b3d957e. This is attempt 156: this message will repeat every 5th retry.
W20260927 23:34:02.835817 5051 consensus_peers.cc:487] T 70cb3313148a47f8b6c692222508eb88 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70cb3313148a47f8b6c692222508eb88. This is attempt 151: this message will repeat every 5th retry.
W20260927 23:34:03.110241 5051 consensus_peers.cc:487] T bbdfdfc23b0c4e3d9224ac2815c6cc65 P f5ec98b2972d44838fa273eb7a10d01a -> Peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Couldn't send request to peer 7e62b4926a8448b2b5f4923553328bca. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bbdfdfc23b0c4e3d9224ac2815c6cc65. This is attempt 151: this message will repeat every 5th retry.
W20260927 23:34:03.677533 5176 master_proxy_rpc.cc:177] Failed to determine new leader Master: Timed out: ConnectToClusterRpc(addrs: 127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749, num_attempts: 754) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
F20260927 23:34:03.690801 5185 master-stress-test.cc:298] Check failed: _s.ok() Bad status: Timed out: Error creating table default.Table_68574d2015dd47bbb92137d783bad572 on the master: CreateTable timed out after deadline expired: CreateTable passed its deadline: Timed out: ConnectToClusterRpc(addrs: 127.2.57.62:35155,127.2.57.61:46527,127.2.57.60:33749, num_attempts: 754) passed its deadline: Network error: Client connection negotiation failed: client connection to 127.2.57.62:35155: connect: Connection refused (error 111)
*** Check failure stack trace: ***
*** Aborted at 1790552043 (unix time) try "date -d @1790552043" if you are using GNU date ***
W20260927 23:34:03.698179 5176 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.57.62:35155)
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e8000008e4) received by PID 2276 (TID 0x7f1b74dfc700) from PID 2276; stack trace: ***
@ 0x2b0af0 __tsan::CallUserSignalHandler() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2b31e0 rtl_sigaction() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
@ 0x7f1b868fb980 (unknown) at ??:0
@ 0x7f1b81bb8e87 gsignal at ??:0
@ 0x7f1b81bba7f1 abort at ??:0
@ 0x2b4f37 __interceptor_abort at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7f1b84becdf4 google::LogMessage::SendToLog() at ??:0
@ 0x7f1b84bed9a0 google::LogMessage::Flush() at ??:0
@ 0x7f1b84bf2adb google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x3572c4 kudu::MasterStressTest::CreateTableThread() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/master-stress-test.cc:?
@ 0x3874ce _ZNSt3__18__invokeIMN4kudu16MasterStressTestEFvvEPNS1_26MasterStressTest_Test_TestEJEvEEDTcldsdeclsr3std3__1E7forwardIT0_Efp0_Efp_spclsr3std3__1E7forwardIT1_Efp1_EEEOT_OS7_DpOS8_ at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/type_traits:3840
@ 0x387381 _ZNSt3__116__thread_executeINS_10unique_ptrINS_15__thread_structENS_14default_deleteIS2_EEEEMN4kudu16MasterStressTestEFvvEJPNS6_26MasterStressTest_Test_TestEEJLm2EEEEvRNS_5tupleIJT_T0_DpT1_EEENS_15__tuple_indicesIJXspT2_EEEE at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:281
@ 0x386acb std::__1::__thread_proxy<>() at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/installed/tsan/include/c++/v1/thread:?
@ 0x2b11b0 __tsan_thread_start_func at /data0/somelongdirectorytoavoidrpathissues/src/kudu/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:959
@ 0x7f1b868f06db start_thread at ??:0
@ 0x7f1b81c9b61f clone at ??:0
I20260927 23:34:03.806599 8077 raft_consensus.cc:482] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:34:03.807117 8077 raft_consensus.cc:504] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } } peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } }
I20260927 23:34:03.809748 8077 leader_election.cc:288] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:34:03.812325 5054 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
W20260927 23:34:03.813005 5051 leader_election.cc:341] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: 9873b2e70c2f49b9be88b3cf8cc6c9b1
I20260927 23:34:03.813356 5051 leader_election.cc:302] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:34:03.814365 8077 raft_consensus.cc:2726] T 9873b2e70c2f49b9be88b3cf8cc6c9b1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:34:03.866529 8077 raft_consensus.cc:482] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:34:03.867079 8077 raft_consensus.cc:504] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
I20260927 23:34:03.870613 8077 leader_election.cc:288] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)
W20260927 23:34:03.871865 5054 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
W20260927 23:34:03.872373 5051 leader_election.cc:341] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: e4e5cb0d645846159e09ab71a897f6df
I20260927 23:34:03.872747 5051 leader_election.cc:302] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:34:03.873673 8073 raft_consensus.cc:2726] T e4e5cb0d645846159e09ab71a897f6df P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:34:03.998986 8077 raft_consensus.cc:482] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 23:34:03.999536 8077 raft_consensus.cc:504] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5ec98b2972d44838fa273eb7a10d01a" member_type: VOTER last_known_addr { host: "127.2.57.3" port: 45519 } } peers { permanent_uuid: "7e62b4926a8448b2b5f4923553328bca" member_type: VOTER last_known_addr { host: "127.2.57.1" port: 40571 } } peers { permanent_uuid: "bad92a10de2c4459827cdbb18854d76d" member_type: VOTER last_known_addr { host: "127.2.57.2" port: 45717 } }
W20260927 23:34:04.007879 5054 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
W20260927 23:34:04.009122 5051 leader_election.cc:341] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571): Not found: Tablet not found: abb39e8b5d2946ceae480a6a6f9cefb1
I20260927 23:34:04.009529 5051 leader_election.cc:302] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5ec98b2972d44838fa273eb7a10d01a; no voters: 7e62b4926a8448b2b5f4923553328bca, bad92a10de2c4459827cdbb18854d76d
I20260927 23:34:04.010610 8086 raft_consensus.cc:2726] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260927 23:34:04.011333 8077 leader_election.cc:288] T abb39e8b5d2946ceae480a6a6f9cefb1 P f5ec98b2972d44838fa273eb7a10d01a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e62b4926a8448b2b5f4923553328bca (127.2.57.1:40571), bad92a10de2c4459827cdbb18854d76d (127.2.57.2:45717)